Unable to enlarge nvs partition

[wijnsema]

For my project with ESP32 WROOM32-E 4MB (Chip is ESP32-D0WD-V3) I need a larger non volatile storage.

Currently I'm using this partioning which is working OK, all offsets are automatically calculated by the partitioning tool:

Code: Select all

# Name,   Type, SubType, Offset,   Size, Flags
nvs,      data, nvs,     0x9000,   0x5000,
otadata,  data, ota,     ,         0x2000,
app0,     app,  ota_0,   ,         0x160000,
app1,     app,  ota_1,   ,         0x160000,
phy_init, data, phy,     ,         0x1000

But as soon as I exceed the 0x5000 for the nvs partion, which in turn will offset the app0 ota_0 partition to 0x20000 (in stead of 0x10000) the ESP32 will get into a reboot loop.

So for example this table:

Code: Select all

# Name,   Type, SubType, Offset,   Size, Flags
nvs,      data, nvs,     0x9000,   0xa000,
otadata,  data, ota,     ,         0x2000,
app0,     app,  ota_0,   ,         0x160000,
app1,     app,  ota_1,   ,         0x160000,
phy_init, data, phy,     ,         0x1000

will lead to a reboot loop like this:

Code: Select all

rst:0x3 (SW_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:10100
ho 0 tail 12 room 4
load:0x40080400,len:5828
entry 0x400806a8
ets Jul 29 2019 12:21:46

rst:0x3 (SW_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:10100
ho 0 tail 12 room 4
load:0x40080400,len:5828
entry 0x400806a8
ets Jul 29 2019 12:21:46

etc.....

So how can I enlarge my nvs?

Help would be greatly appreciated.

Kind regards, Ronald Wijnsema




BTW I'm using PlatformIO for development:
Checking platformio/contrib-pysite 2.37.191020 @ ~2.37.0 [Incompatible 2.39.201019]
Checking platformio/tool-unity 1.20500.200612 @ ~1.20500.0 [Up-to-date]
Checking platformio/tool-scons 4.40100.2 @ ~4.40100.2 [Up-to-date]

Platform Manager
================
Platform espressif32
--------
Checking platformio/espressif32 2.1.0 [Outdated 3.2.0]
Checking platformio/toolchain-xtensa32 2.50200.80 @ ~2.50200.0 [Outdated 2.50200.97]
Checking platformio/toolchain-esp32ulp 1.22851.191205 @ ~1.22851.0 [Up-to-date]
Checking platformio/tool-esptoolpy 1.30000.201119 @ ~1.30000.0 [Up-to-date]
Checking platformio/tool-mkspiffs 2.230.0 @ ~2.230.0 [Up-to-date]
Checking platformio/tool-cmake 3.16.4 @ ~3.16.0 [Up-to-date]
Checking platformio/tool-ninja 1.9.0 @ ^1.7.0 [Up-to-date]
Checking platformio/tool-mconf 1.4060000.20190628 @ ~1.4060000.0 [Up-to-date]
Checking platformio/tool-idf 1.0.1 @ ~1.0.1 [Up-to-date]

Platform espressif32
--------
Checking git+https://github.com/platformio/platform-espressif32.git 2.1.0+sha.a58a358 [Outdated 2.1.0+sha.56f9c60]
Checking platformio/toolchain-xtensa32 2.50200.80 @ ~2.50200.0 [Outdated 2.50200.97]
Checking platformio/toolchain-esp32ulp 1.22851.191205 @ ~1.22851.0 [Up-to-date]
Checking platformio/tool-esptoolpy 1.30000.201119 @ ~1.30000.0 [Up-to-date]
Checking platformio/tool-mkspiffs 2.230.0 @ ~2.230.0 [Up-to-date]
Checking platformio/tool-cmake 3.16.4 @ ~3.16.0 [Up-to-date]
Checking platformio/tool-ninja 1.9.0 @ ^1.7.0 [Up-to-date]
Checking platformio/tool-mconf 1.4060000.20190628 @ ~1.4060000.0 [Up-to-date]
Checking platformio/tool-idf 1.0.1 @ ~1.0.1 [Up-to-date]

[ESP_Minatel]

Hi,

Your partition mapping seems to be ok. I've tested here using ESP-IDF 4.4 (master) and the partition find example.

Example: https://github.com/espressif/esp-idf/tr ... ition_find

I got no reset when increasing the nvs partition.

[ESP_Minatel]

Here is the log output:

Code: Select all

I (0) cpu_start: App cpu up.
I (247) cpu_start: Pro cpu start user code
I (247) cpu_start: cpu freq: 160000000
I (247) cpu_start: Application information:
I (252) cpu_start: Project name:     partition_find
I (257) cpu_start: App version:      1
I (261) cpu_start: Compile time:     May 17 2021 16:10:44
I (268) cpu_start: ELF file SHA256:  df7379e64fbca8bd...
I (274) cpu_start: ESP-IDF:          v4.4-dev-1395-gec4d67ae4a-dirty
I (281) heap_init: Initializing. RAM available for dynamic allocation:
I (288) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (294) heap_init: At 3FFB2B90 len 0002D470 (181 KiB): DRAM
I (300) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (306) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (313) heap_init: At 4008A778 len 00015888 (86 KiB): IRAM
I (320) spi_flash: detected chip: gd
I (323) spi_flash: flash io: dio
W (327) spi_flash: Detected size(8192k) larger than the size in the binary image header(4096k). Using the size in the binary image header.
I (341) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (351) example: Printing partition table csv file contents for reference...

# Name,   Type, SubType, Offset,   Size, Flags
nvs,      data, nvs,     0x9000,   0xa000,
otadata,  data, ota,     ,         0x2000,
app0,     app,  ota_0,   ,         0x160000,
app1,     app,  ota_1,   ,         0x160000,
phy_init, data, phy,     ,         0x1000
I (381) example: ----------------Find partitions---------------
I (391) example: Find partition with type ESP_PARTITION_TYPE_DATA, subtype ESP_PARTITION_SUBTYPE_DATA_NVS, label NULL (unspecified)...
I (401) example: 	found partition 'nvs' at offset 0x9000 with size 0xa000
I (411) example: Find partition with type ESP_PARTITION_TYPE_DATA, subtype ESP_PARTITION_SUBTYPE_DATA_PHY, label NULL (unspecified)...
I (421) example: 	found partition 'phy_init' at offset 0x2e0000 with size 0x1000
I (431) example: Find partition with type ESP_PARTITION_TYPE_APP, subtype ESP_PARTITION_SUBTYPE_APP_FACTORY, label NULL (unspecified)...
E (441) example: 	partition not found!
I (451) example: Find partition with type ESP_PARTITION_TYPE_DATA, subtype ESP_PARTITION_SUBTYPE_DATA_FAT, label NULL (unspecified)...
E (461) example: 	partition not found!
I (461) example: Find second FAT partition by specifying the label
I (471) example: Find partition with type ESP_PARTITION_TYPE_DATA, subtype ESP_PARTITION_SUBTYPE_DATA_FAT, label storage2...
E (481) example: 	partition not found!
I (491) example: ----------------Iterate through partitions---------------
I (491) example: Iterating through app partitions...
I (501) example: 	found partition 'app0' at offset 0x20000 with size 0x160000
I (511) example: 	found partition 'app1' at offset 0x180000 with size 0x160000
I (521) example: Iterating through data partitions...
I (521) example: 	found partition 'nvs' at offset 0x9000 with size 0xa000
I (531) example: 	found partition 'otadata' at offset 0x13000 with size 0x2000
I (541) example: 	found partition 'phy_init' at offset 0x2e0000 with size 0x1000
I (551) example: Example end

[wijnsema]

Thank you very much for your quick reply Minatel!

I will have to investigate further then. Two possibilties seems to be remaining:

• This is caused by me using the Arduino stack which is based on ESP-IDF 4.01 (I believe)
• Or this is caused by the Platform I/O environment

Which one is most likely you think?

Thanks!

[ESP_Minatel]

Hi,

Not sure about the root cause. You can test with the ESP-IDF without PIO.

[ESP_jakob]

Hi wijnsema,

did you flash all the new partitions to your board? Or have you tried to do a full erase of the flash? If one of the mentioned actions fixes the problem, probably something went wrong during the switch to the new partition table. But let's first see whether that's the case.

Best,
Jakob

[chegewara]

Hi,
i think i can answer question why you cant increase size on NVS partition. Today ive been strugling with partitions problem in pio and i found something. PIO has hardcoded esptool script i think, because it is flashing app always at offset 0x10000. I mean always.
I did few experiments and here are last 2 steps that leads to that conclusion:
- STEP 1: flash test app with esp-idf and this example of partitions table:

Code: Select all

# Name,   Type, SubType, Offset,  Size, Flags
# Note: if you have increased the bootloader size, make sure to update the offsets to avoid overlap
nvs,        data, nvs,      0x9000,  0x6000,
phy_init,   data, phy,      0xf000,  0x1000,
spiffs,   data, spiffs,  , 0x30000,
factory,    app,  factory,  0x100000, 1M,
storage1,    data, fat,             , 0x40000,
storage2,    data, fat,             , 0x40000,

- STEP 2: flash with PIO and the same partition table, but with this example app:

Code: Select all

#include <Arduino.h>

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
}

void loop() {
  // put your main code here, to run repeatedly:
  Serial.println("test");
  delay(1000);
}

You can imagine how much surprised i was when i saw output log from esp-idf app instead of "test" every 1 second.

My suggestion is to put NVS partition at the end of partition table, which will fix you issue for sure.

[wijnsema]

I investigated things further and the problem occurs with the Arduino core for ESP32. It is not directly related to PlatformIO because I could reproduce the problem in the Arduino IDE, however I manually created an other partition table because the precoded tables were not suitable for me. As soon as the 0x10000 was exceeded, the ESP came in a reboot loop. It seems to be caused by PlatformIO because PIO lets you change the partitions so easy!

@chegewara is right, the partition tooling is hardcoding the first factory/ota at 0x10000. This article from 2018 is mentioning the same problem:
https://desire.giesecke.tk/index.php/20 ... n-esptool/

I played around with the solutions that were suggested in this article but couldn't get it working. Also switching to the master branch of the https://github.com/espressif/arduino-esp32/ code did not solved the problem. The 0x10000 is still hard coded in there.

When I have time I will file a bug report at the arduino-esp32 repo.

For now I am using a work-around, like @chegewara suggested, to put the nvs at the end of the table.

@ESP_jacob yes I did a full flash erase all the time before changing the partitions, just to be sure.

Ronald Wijnsema