Esp32S3 : Why time to write in Flash is so long ? Help !

[ThomasESP32]

Good morning,

I am using an Esp32S3 and I have a problem when writing something in Flash memory.
Could you please help me ?

1) 1st thing I have done :

I defined a partition type FAT in flash memory.
I mounted a FileSystem (Type FAT FileSystem) on this partition using Virtual File System component.
My goal was to write something (10 bytes) in a file as fast as possible.

The problem I saw was that in order to write something in my file, I had to open it, write something in it and close it.
I didn't manage to flush the things I had write without closing the file. And all these operations (Open, write, close) tooks
something like 2 seconds. This was too much time for the application.

Question 1 : Do you know why it tooks 2s to write 10 bytes in my file in this case please ?
Question 2 : Have you already managed to flush the things you had writen in a file without closing it (Using FAT FileSystem and VFS component) ? This would enable me to write things in my file without opening and closing it. => Maybe it would be faster ?

2) The second thing I have done :

Because this operation tooks too much time, I decided to write the 10 bytes in an NVS.
So, I have created a special partition, I have mounted an NVS on the partition and I am using the set_blob methods in order to write the 10 bytes in the partition. The amount of time needed by this operation is sometime 150ms, but sometime 4700ms.
I have 2 other NVS opened on two other partitions.
Could you please explain me why it takes this big amount of time in order to write something on my NVS please ?


Maybe the solution would be to write things in a file again but flushing the datas in it instead of opening and closing
the file each time. Have you already done it ? Do you have any example please ?

Thank you for your help, this is really important.
I would like to write my 10 bytes of datas in about 150ms each time.
Do you have any idea please ?

Best regards,

Thomas TRUILHE

[MicroController]

Question 1 : Do you know why it tooks 2s to write 10 bytes in my file in this case please ?

Don't know. But you can try SPIFFS instead of FAT which should incur less overhead in terms of flash reads/writes per write operation.

Question 2 : Have you already managed to flush the things you had writen in a file without closing it (Using FAT FileSystem and VFS component) ?

Try using f_sync().

Or, don't use a file system at all and just write your data to a bare partition. If you actually only have one file, where you keep appending data, that approach would be pretty straight-forward to implement and as fast as it gets.

[ThomasESP32]

Do you have any example in order to use f_sync please ?
I have tried it but I didn't manage to sync the datas in the file.

Moreover, what do you mean about using bare partition. Do you think about
calling esp_flash_partition methods ?

Thank you.

[ThomasESP32]

Moreover I have a problem. Could you please help me ?

As I understand, f_sync() is part of the FAT FileSystem library.

In my application, I have 3 different FileSystem registered using Virtual FileSystem.
At the moment, I am using the standard C functions in order to access to the different file.
Virtual FileSystem component manages the use of the FAT FileSystem methods (f_open, f_read, f_write...)
depending on the C functions called.

How can I use directly the FAT FileSystem methods (In order to use f_sync()).
I am trying to do a f_open() using FATFileSystem function instead of stand C function
but the method does not manage to open the file...

Maybe it is because I have many filesystem ?
Is there a way to call f_sync() using Virtual FileSystem functions ??

Best regards,

[RalphD]

Dear friend as other have already mentioned fatfs is the wrong way to go with the flash memory. The overhead is far too much. The Xtensa LX7 MCU connects to flash memory in the best case via 8bit SPI sometimes 4bit depending on your package type. Not the reason why it is so slow. However, Flash memories in general can only be written using always pages. Depending on the Flash Chip these pages are 4, 8, 16 or even 32kB. If you write only 10 bytes in the same file, that means that any underlying code needs to deal with a FULL page, smallest 4kB. As the code does not know what other data in the other part of the flash page is, it copies and merges back and force possible 12kB or more for writing just 10 bytes. So there is no way for speeding anything up on small data.

Not to mention that write cycles of flash memory are limited.

My suggestion stay away from FATFS

[ThomasESP32]

Ok thank you for your answer.
However, I have tried fsyc(fileno(FileStream)) but this does not flush the datas correctly in my file.

Any suggestion in order to deal with 10 bytes at each write ????

What can I do please ?

[ThomasESP32]

Let's say I have two instances of FAT FileSystems.
The 1st one has the path : /flash1
The 2nd one has the path : /flash2

In order to open a file on FileSystem1, I can do : FILE *MyFile = fopen("/flash1/readme.txt", "a+b");
In order to write something in this file I can do : fwrite("Hello\0", 1, strlen("Hello\0"), MyFile);

But how can I do an flush on the file please ?

[RathiSonika]

Question 2 : Have you already managed to flush the things you had writen in a file without closing it (Using FAT FileSystem and VFS component) ? This would enable me to write things in my file without opening and closing it. => Maybe it would be faster ?

You can check a config option called "Enable automatic f_sync" in the Component Config under FAT Filesystem support (idf.py menuconfig-> Component config -> Fat Filesystem support -> Enable automatic f_sync). Normally, this option is disabled. But if you enabled it, it will ensure automatic calling of f_sync() to flush recent file changes after each call of write.

But how can I do an flush on the file please ?

you can use
FILE *MyFile = fopen("/flash1/readme.txt", "a+b");
fwrite("Hello\0", 1, strlen("Hello\0"), MyFile);
fflush(MyFile);