Storing binary (Uint8Array) to disk with React Native and Zustand

The tragedy of using react native async-storage with automerge and zustand

The last couple of weeks I have spent getting my head around the automerge library to be able to use CRDTs.

It all looked really simple and awesome. I coded along the example, and extended it to fit my future use cases.

After somewhat understanding it, I felt ready to use it inside my React Native project. Little did I know...

The first argument to Automerge.change must be the document root

Again and again.. Something did not work as expected. I won't go into much context, but in between I had to store a Uint8Array to disk and load that Uint8Array again.

Storing and loading the Uint8Array in my proof of concept in the browser using "localForage" package worked like a charm. But "@react-native-async-storage/async-storage" requires values to be stored as strings. I did expect that was the same case for localForage, a package for storing to localstorage but little did I know... I assume they use something smarter than JSON.stringify() and JSON.parse() when it comes to storing/loading UInt8Arrays (But I did not investigate that part further).

Since the storing and loading of the Uint8Array kept throwing weird errors in my react-native project, i decided to spin up the node REPL. Quite quickly, the problem became obvious:

> x = new Uint8Array([1,2,3])
Uint8Array(3) [ 1, 2, 3 ]
> x
Uint8Array(3) [ 1, 2, 3 ]
> y = JSON.stringify(x)
'{"0":1,"1":2,"2":3}'
> y
'{"0":1,"1":2,"2":3}'
> z = JSON.parse(y)
{ '0': 1, '1': 2, '2': 3 }
> z
{ '0': 1, '1': 2, '2': 3 }

Yeah, Uint8Array can not be converted with JSON.stringify and JSON.parse.... Which sadly was implizitly done automaticly by my (at that time) persitance library I added to my state management library. Automaticly persisting global state in localstorage calling JSON.parse and JSON.stringify on everything in the store. WHich is great, don't get me wrong. But it kind of sucks if that throw to weird, mind bogling errors. I used zustand with the persist middleware. Therefore I decided to just drop the persist middleware and made storing to disk an explizit task again.

What I had to do was simply just create a function to store my Uint8Array to disk as a string and load it again, without losing the information.

Steps taken to convert Uint8Array to String and back to Uint8Array

  • make sure to install "buffer to have it available inside react-native
    • npm i --save buffer
  • Code:
    const u8 = new Uint8Array([65, 66, 67, 68]);
    const b64FromU8 = Buffer.from(u8).toString('base64');
    const u8FromB64 = new Uint8Array(Buffer.from(b64, 'base64'))
    
    and that is literally all it took to fix the issue.. 🙃

Takeaways

  • Automaticly persisting your state is great, if JSON.parse and JSON.stringify is enough.
  • Implementing explizit state persistance was no pain at all. Think twice if you want to use a function/library/whatever for this.
  • Base64 is used to convert from string to binary and vice versa.
  • If you are not using it, I still enjoy using zustand for global state management.

And that's the end for now, I wish you a nice day!