Data Transformation
Somebody rebuilds this by hand every month
Data transformation is turning that manual routine into an automated, repeatable process — one that produces the same output every time, lists what it could not resolve, and belongs to your team rather than to us.
By hand
Export from ERPPaste into templateFix the dates by handLook up 40 missing codesPivot, check, email≈ 2 days, once a monthAutomated
Same exportRules run automaticallyExceptions listed for reviewOutput in the agreed formatAudit trail of what changed≈ 20 minutes, on demandWhat we usually find
Every business has three or four of these
They are rarely on anyone's project list, because each one is only a couple of days. Added up across a year, they are somebody's whole job.
The monthly pack
Two days of copying, pivoting and checking, every month, by someone whose time is worth considerably more.
The system bridge
An export from one system reshaped by hand so another system will accept it.
The customer file
A format a major customer insists on, produced manually because it was easier than asking.
The compliance return
A submission assembled from four sources, where a mistake is expensive and nobody has time to check twice.
The reconciliation
Two reports compared line by line to find the handful of rows that disagree.
The ad hoc that stuck
A one-off analysis somebody now expects every fortnight.
The change
Two days becomes twenty minutes
The time saved matters less than the consistency. The same input produces the same output, every time, and you can see what it did.
By hand
Export from ERPPaste into templateFix the dates by handLook up 40 missing codesPivot, check, email≈ 2 days, once a monthAutomated
Same exportRules run automaticallyExceptions listed for reviewOutput in the agreed formatAudit trail of what changed≈ 20 minutes, on demandThe person who used to do it does not lose the job. They lose the part of it nobody wanted.
The rules
Turn the data you have into the data you need
Split or combine fields, standardise values, convert formats, map old codes to new, align classifications, build the import structure another system demands. Each rule is written down and agreed before it runs.
CUST_NMCustomer nameTrim, title case, strip trading-name suffixADDR1 + ADDR2 + ADDR3Street / Suburb / State / PostcodeSplit and validate against Australia PostPROD-CD (legacy)Item code (new)Cross-reference table, 4,182 rowsTYPE = G / C / IGeneral / Commercial / IndustrialLookup. Unknown values escalatedDATE_STR (12 formats)ISO 8601 dateParse by pattern. Ambiguous dates escalatedQuestions
Transformation, answered
Data transformation is converting information from the structure you have into the structure you need. Splitting or combining fields, standardising values, mapping old codes to new ones, aligning classifications, consolidating datasets and building the import structures another system requires.
Usually yes, and it is one of the most common jobs we take. The manual work is normally a sequence of decisions somebody has memorised. Once those decisions are written down as rules, the transformation runs on demand and the two days disappear.
Rules change constantly, which is why they are documented and editable rather than buried in a script nobody can read. Changing a rule is a change to the rule, not a rebuild of the process.
Talk to us