Unable to compress strings

[alkhachatryan]

I'm using miniz.h to compress a simple string with ONE letter.

Code: Select all

  // Example string to compress
    const char* originalString = "T";
    size_t originalLength = strlen(originalString) + 1; // +1 for null terminator

    // Allocate memory for compressed data
    mz_ulong compressedLength = compressBound(originalLength); // Use mz_ulong
    Serial.println("Free size:");
    Serial.println(esp_get_free_heap_size());
    Serial.println("Needed size:");
    Serial.println(compressedLength);

    unsigned char* compressedData = (unsigned char*)malloc(compressedLength);
    if (compressedData == NULL) {
        Serial.println("Failed to allocate memory for compressed data");
        return;
    }

    // Compress the string
    int status = compress(compressedData, &compressedLength, (const unsigned char*)originalString, originalLength);
    if (status != Z_OK) {
        Serial.print("Compression failed with status: ");
        Serial.println(status);
        free(compressedData);
        return;
    }

    Serial.print("Compressed length: ");
    Serial.println(compressedLength);

    // Allocate memory for decompressed data
    unsigned char* decompressedData = (unsigned char*)malloc(originalLength);
    if (decompressedData == NULL) {
        Serial.println("Failed to allocate memory for decompressed data");
        free(compressedData);
        return;
    }

    // Decompress the string
    mz_ulong decompressedLength = originalLength; // Use mz_ulong
    status = uncompress(decompressedData, &decompressedLength, compressedData, compressedLength);
    if (status != Z_OK) {
        Serial.print("Decompression failed with status: ");
        Serial.println(status);
        free(compressedData);
        free(decompressedData);
        return;
    }

    Serial.print("Decompressed length: ");
    Serial.println(decompressedLength);

    // Print the decompressed string
    Serial.print("Decompressed string: ");
    Serial.println((char*)decompressedData);

    // Free allocated memory
    free(compressedData);
    free(decompressedData);

So every time I get -4 error telling that there is an issue of memory. But the debug shows me the following:

Free size:
298036
Needed size:
135
Compression failed with status: -4

I cannot understand what's wrong with it.

Hardware
ESP32 Wroom 32u

IDE
VSCode + PlatformIO

platformio.ini

Code: Select all

[env:esp32dev2]
platform = espressif32
board = nodemcu-32s

framework = arduino, espidf    
upload_port = /dev/ttyUSB1
monitor_port = /dev/ttyUSB1
monitor_speed = 115200
lib_extra_dirs = /home/alexey/Projects/DTLP/DTLP
upload_speed = 921600

; Configure flash settings
board_build.f_flash = 40000000
board_build.flash_mode = qio
board_build.flash_size = 4MB
board_build.cpu_freq = 240MHz
board_build.partitions = partitions.csv
board_build.mcu = esp32
board_build.psram = one

board_upload.flash_erase = true


; Configure events to run on core 1 (if needed, this is more for multi-core setups)
build_flags = 
    -D ARDUINO_RUNNING_CORE=1
    -Ilib/miniz
    -Ilib/heatshrink
    -DBOARD_HAS_PSRAM

partitions.csv

Code: Select all

# Name,	Type,	SubType,	Offset,	Size,	Flags
nvs,	data,	nvs,	0x9000,	0x5000,	
otadata,	data,	ota,	0xe000,	0x2000,	
app0,	app,	ota_0,	0x10000,	0x140000,	
app1,	app,	ota_1,	0x150000,	0x140000,	
spiffs,	data,	spiffs,	0x290000,	0x160000,	
coredump,	data,	coredump,	0x3f0000,	0x10000,	

So again, everything shows I have enough memory to compress a single "T" letter, but I have an error while trying to allocate memory for that.

[ESP_Sprite]

miniz needs a bunch of space to allocate data structures that are needed for compression, e.g. for the dictionary. Could be that you don't have enough (contiguous) memory for that, making the call fail.

[alkhachatryan]

So maybe I need to play with partitions? What to do for this case? My chip is esp32 wroom 32u. What to enable/configure? If there is miniz lib for esp32 - we must to be able to use. Otherwise it's garbage. So the question is: what to do to use it?

[ESP_Sprite]

We're not talking about flash here, we're talking about RAM. And I'm sorry, but I can't tell you exactly what is going wrong - 300K seems enough RAM to me, but perhaps your RAM is fragmented and miniz tries to allocate a large contiguous buffer.

[alkhachatryan]

I didn't do anything from my side to make fragments. Can you tell me please what and where to check?

What I need - just send signals to digital pins, read from them and use compression, that's it. I send data through laser and just want to compress before sending

[MicroController]

Can you share some more code to help us narrow down your issue?