Best library/API to use for custom nonvolatile database

[chris.f.rice]

Hello, we are getting started using an ESP32c6, and we would like to implement some light custom database functionality, using nonvolatile storage.

I see that the NVS library is available, but that it's "designed to store key-value pairs in flash."

I think that's (key value pairs) not exactly the model we are going for, we would actually like to access non-volatile storage directly, defining our own tabular data layout and accessing locations with reads and writes as the calling functions request to store and retrieve different types of data.

Is there a different library we should work with? Or some example code that is close to what we are trying to do?

Thank you!

[MicroController]

You may want to reconsider the approach you're envisioning now.
Note that generally data in flash memory cannot be updated arbitrarily. You cannot update a '0' bit to a '1' bit without first erasing the whole 4kb page containing the bit. Usually, you also want to avoid erasing/writing the same page repeatedly (-> "wear levelling") which implies that the location/address of a piece of data needs to change with every update to the data.
You can look into the NVS component, which internally basically operates via copy-on-write, or look into one of the file systems available (SPIFFS, LittleFS, FAT) which also implement the flash page management for you.

[chris.f.rice]

Thank you MicroController. I do get that, that FLASH memory generally requires erasing a page at a time, and we have some reusable roll-your-own code that manages that (copies page out, edits, erases original page, writes modified page). I will definitely look into the libraries you mention (SPIFFS, LittleFS, FAT)... but we may decide not to do it filesystem style as well.

If we end up wanting to use a custom approach and just use a random access (page based, as you reminded me thank you) FLASH directly, does ESP-IDF provide an API for this, or would we just end up accessing a hardware peripheral directly?

Thanks again!

[MicroController]

You may be able to leverage the wear levelling component of the IDF.

[ESP_jakob]

If you want to have low-level access to the flash, it's probably best to use the ESP-Partition (https://docs.espressif.com/projects/esp ... ition.html) API. But then you really need to take care of proper wear leveling yourself.