Automatically migrate JSON data to newest version of JSON schema-CodePudding

I have a service running on my linux machine that reads data stored in a .json file when the machine is booting. The service then validates the incoming JSON data and modifies specific system configurations according to the data. The service is written in C and for the validation im using https://github.com/pboettch/json-schema-validator.

In development it was easy to modify the JSON schema and just adapt the data manually. I've started to use semantic versioning for my JSON schema and included it the following way:

JSON schema:

{
  "$id": "https://my-company.org/schemas/config/0.1.0/config.schema.json",
  "$schema": "http://json-schema.org/draft-07/schema#",
  // Start of Schema definition
}

JSON data:

{
  "$schema": "https://my-comapny.org/schemas/config/0.1.0/config.schema.json",
  // Rest of JSON data
}

With the addition of the version, I am able to check if a version mismatch exists before validating.

What I am looking for is a way to automatically migrate the JSON data to match the newer schema version, if a version mismatch is identified. Is there any way to automatically achieve this, or is the only way to manually edit the JSON data to match the schema?

Since I plan on releasing this as open source I would really like to include some form of automatic migration so I can just ask the user if he wants to migrate to conform to the newest schema version instead of throwing an error, if a version mismatch was identified.

CodePudding user response：

What you're asking for is something which will need to make assumptions to work.

This is an age old problem and similar for databases. You can have schema migrations generated with many simple changes, but this is not viable if you wish to translate existing data automatically too.

Let's look at a basic example. You rename a field. How would a tool know you've renamed a field vs removed an old one and added a new one? It essentially, cannot.

So, you need to write your migrations by hand.

You could use JSON transformation tools like jq or fx to create migration scripts without writing it in code, which may or may not be preferable. (jq has a steeper learning curve but it's also very powerful.)