Decoding mjpg stream to esp32

[johnbchron]

Hello all! I have a project idea but I'm not sure how to go about making it.

I have an octopi setup and it is currently sending an mjpg stream over wifi. Right now my esp32 can connect to wifi and is running an ST7789 tft display (runs over spi) through the wonderful TFT_eSPI library. I am trying to figure out how to stream the contents of the mjpg stream onto the display of my esp32. The resolution is not what I'm going for here, it can be super low but I would at least like to get about 5 fps. I can lower the stream's resolution to whatever is needed to decrease the computational load on the esp32.

I don't know where to start. I don't know how to request/decode a jpg, much less a stream of them. I have some idea of how to display a decoded jpg using Bodmer's jpg decoding library though...

If you have some idea of how to do this, please tell me at least what direction to go to try to start this... really any information would be helpful. I've tried googling similar projects but the results get drowned out with the 1000's of esp32cam tutorials

Anyways, thanks in advance for any help you offer. Really, I appreciate it.

[johnbchron]

I'm replying to my own post to make it more recent in the feed... I really would like to figure this out so even if you just have a thought or maybe someplace to point me to, please post it here.

[tuupola]

ESP32 is capable of playing 320x240 MJPEG video approximately 8-9 fps. This probably could be tuned up to 12 fps. I made a proof of concept video player which plays RAW RGB565 and MJPEG videos from sd card. You could probably use that as a starting point. It is ESP-IDF but probably could be ported to Arduino. See the demonstration video in Vimeo, code can be found in GitHub.

https://vimeo.com/409435420
https://github.com/tuupola/esp_video

Edit: I also did a short write up which explains the code bit better.

https://appelsiini.net/2020/esp32-mjpeg-video-player/