Refactoring a Company ERP System Without Losing Existing Data
Introduction: Why Refactor?
Once a company grows to a certain scale, it will usually introduce an ERP system to manage materials, products, and accounting. From introducing the system and building its data to helping employees become familiar with it, the adjustment process often takes years.
The software industry, however, moves quickly. This has been especially noticeable in recent years: packages and frameworks may receive a minor release every month and a major release every six months. A workflow may not even be fully established before another new framework appears.
Even within the same company, different systems may be introduced at different times to meet different needs. One system may have arrived in 2000 and another in 2010. Eventually, the result becomes a jumble of systems: every department does its own thing, there is no unified architecture, and the data is either messy or left unmanaged.
But leaving an old system alone simply because it still runs is not a solution. Without long-term maintenance, security risks continue to accumulate, while the system's reliability and correctness may gradually decline, like an aging machine slowly gathering rust.
The real challenge is therefore learning how to split up the system's responsibilities. Instead of replacing everything at once, we refactor the old system one step at a time, or replace individual functions with systems that can be upgraded and maintained. This has become essential work for ERP engineers.
A Common ERP Background
In its early years, a company may not have professional programmers managing its ERP system, even though the system still needs to be customized around company data and workflows. Perhaps one process needs an additional document number, or a particular customer receives a special discount.
These requests are often maintained by employees in related departments who know a little about technology, or by engineers whose experience does not fully match the work. Technical debt is difficult to avoid. After years of accumulation, the result can become error-prone and extremely difficult to maintain.
Introducing a new system at this point requires extensive customization because the company has already accumulated so much legacy data. The cost can be enormous, and so can the risk.
Refactoring the entire system is no easier. It is difficult to find one person who understands the business workflows, software architecture, and data structures well enough to coordinate everything. Companies outside the software industry also struggle to estimate the required staff and technical threshold, and it is not easy to find an engineer willing to accept the overall risk. For an individual, this is often not just about compensation. The risk itself is simply too high.
Skills Required for Refactoring
So what abilities are actually needed to refactor an old ERP system? Based on several years of refactoring experience, I believe the following are important:
- Software engineering and the ability to independently plan a maintainable system: this is essential when building an ERP system that must remain maintainable over the long term.
- The ability to distinguish the strengths and weaknesses of frontend and backend frameworks: we do not need to reinvent the wheel, but we do need to know which wheel belongs on which road.
- The ability to analyze company workflows: if the system does not fit the company's actual operations, it will eventually become nothing more than decoration.
- The ability to analyze legacy data structures: data structures are not always what they appear to be. They are usually tied to decisions made when the original system was designed. Understanding the old data is necessary if that data must continue to be used.
The Refactoring Process
1. Inventory Workflows and Requirements
Whether a system is usable depends on whether it matches real business requirements. The first important step is therefore to inventory every essential workflow in the company.
Analyzing those workflows also reveals the company's operational priorities and the habits of its users. Only after counting the workflows and understanding their characteristics and pain points can the system solve real problems and create value for the company.
2. Organize the Legacy Data Structure
A company cannot discard its existing data merely to accommodate a new system. Preserving legacy data is therefore an important prerequisite.
For relatively simple structures, such as basic material or customer data, it may be reasonable to continue using the existing tables temporarily. This also avoids data synchronization problems while both systems run in parallel. The tables can be reconsidered later, when a real problem appears or after the important workflows have been completed.
Reality is rarely so cooperative. Sometimes the legacy system has complicated data sources because functions from different systems were merged together. Sometimes the format is simply old, such as storing everything in one string and defining characters 1 through 10 as one field and characters 11 through 20 as another.
In cases like these, properly structured new tables are needed to normalize the legacy data and make later development manageable.
3. Establish the Foundation and Development Rules
By this point, we understand enough about the company's data and operations to define the system boundary. How much data must the system process each month: hundreds of records or tens of thousands? What reports must it produce: numeric reports or statistical charts? What kinds of users will access it: internal or external users, on phones or computers? All of these conditions affect the choice of frameworks and technology.
Once the frameworks and technology have been chosen, the next step is to establish development rules. In addition to the usual requirements specifications, naming conventions, and architectural layers, it can now be useful to define AI Rules for AI assistants so that collaboration remains consistent.
4. Set the Refactoring Order
Time and staffing are limited. If we could rebuild the entire system in one pass, there would probably be no need to read this article today.
We therefore need to distinguish what is urgent from what can wait, then set the development order according to the available people and time. That is how a large refactoring effort gets completed one step at a time. These are some of my personal priorities:
- Functions that already exist but no longer meet requirements, especially those that may cause data errors that are difficult to repair, usually come first.
- If no system workflow exists and the data is still handled manually, prioritize it when that data affects other workflows. Getting the data into the system makes later work easier.
- If no system workflow exists but the work is simple and can produce a quick result, mix it into the schedule to earn some visible progress.
- Leave functions that the old system still supports reliably until the end. In most cases, the old system can keep carrying that work until the more urgent parts are complete.
5. Migrate and Test Gradually
Once the schedule is set, the rest is the familiar process: develop, test, deploy, validate, gather user feedback, and fix bugs.
Before a new workflow has been tested and proven stable, it is best to keep the old workflow available. If the new system encounters a temporary problem, the old workflow can still serve as a fallback. Once the new workflow has been validated and is stable, the old one can be retired, completing the refactoring of that particular process.
Conclusion
Refactoring is not simple because it touches too many areas: understanding the old system, analyzing the database, interviewing people about their workflows, and designing the technical architecture of the new system. These responsibilities would ideally belong to specialists in different fields, yet they are often assigned to only one or two people.
Is that reasonable? Of course not. But reality does not always cooperate.
That is why I wrote this article: to provide a reference for those who were forced to become a one-person army, and those determined to become one.
English translation provided by Codex.