GROQ / Sanity data handling tips

There are many ways to migrate data, or edit multiple documents from you dataset.

One is to use the client, fetch, and mutate the content in a transaction.

This always feels a little bit 😬 for me. Especially when working with live data.

Alejandro pointed me in the direction of a different, more elegant method the other day:

To use the Sanity CLI to download a specific document type as NDJSON. Manipulate that single file, then upload it using the —replace flag.

The docs also suggest using a tool like ndjson-cli to manipulate data.

Download/export the data:

sanity dataset export production production.tar.gz --types person,page

Manipulate it in your favourite editor, then upload/import it:

sanity dataset import reshapedPerson.ndjson production --replace

This means that you will at least have an overview of the file, and can do JSON validations against it before uploading. A little less 😬 and a little more 😏.

See the Sanity docs for more info on migrations https://www.sanity.io/docs/migrating-data

Creating a migration ndjson to change _type

Requires you to delete and replace dataset.

const fs = require('fs')
const arrayToNdjson = require('array-to-ndjson')
const { documents } = require('./data.json')
const newDocuments = [...documents]
const persons = documents.filter(doc => doc.type === 'person').map(person => {
  const name = person.name
  person.name = { nb: name }
  // delete person.firstName
  // delete person.lastName
  // person._type = 'entity'
  // person.type = 'person'
  const index = documents.findIndex(doc => doc && doc._id === person._id)
  newDocuments[index] = null
  return person
})
// const organizations = documents.filter(doc => doc._type === 'organization').map(org => {
//   org._type = 'entity'
//   org.type = 'organization'
//   if (org.country) {
//     org.countries = [org.country]
//     delete org.country
//   }
//   const index = documents.findIndex(doc => doc && doc._id === org._id)
//   newDocuments[index] = null
//   return org
// })
console.log(documents.length)
console.log(newDocuments.filter(Boolean).length)
const entities = [...newDocuments.filter(Boolean), ...persons]
arrayToNdjson(entities).pipe(fs.createWriteStream('entities.ndjson'))