RNG Quality (Dieharder Test Results Failing)

Post Reply
bnp117
Posts: 17
Joined: Fri Jun 15, 2018 3:08 am

RNG Quality (Dieharder Test Results Failing)

Postby bnp117 » Tue Oct 08, 2019 3:09 am

I am using

Code: Select all

esp_random()
and my ESP32's radios are on (ESP32 uses the radio subsystems for entropy). I collected 16GB of RNG data (streamed over UART at 230400baud) over a period of 8 days and ran the dieharder RNG test-suite on the resulting file using the command

Code: Select all

dieharder -a -g 201 -f rng.raw
The output of the tests are as follows:

Code: Select all

#=============================================================================#
#            dieharder version 3.31.1 Copyright 2003 Robert G. Brown          #
#=============================================================================#
   rng_name    |           filename             |rands/second|
 file_input_raw|                         rng.raw|  4.37e+07  |
#=============================================================================#
        test_name   |ntup| tsamples |psamples|  p-value |Assessment
#=============================================================================#
   diehard_birthdays|   0|       100|     100|0.82477712|  PASSED  
      diehard_operm5|   0|   1000000|     100|0.16845445|  PASSED  
  diehard_rank_32x32|   0|     40000|     100|0.11779202|  PASSED  
    diehard_rank_6x8|   0|    100000|     100|0.18180463|  PASSED  
   diehard_bitstream|   0|   2097152|     100|0.00000000|  FAILED  
        diehard_opso|   0|   2097152|     100|0.00000000|  FAILED  
        diehard_oqso|   0|   2097152|     100|0.11056558|  PASSED  
         diehard_dna|   0|   2097152|     100|0.78492893|  PASSED  
diehard_count_1s_str|   0|    256000|     100|0.00000000|  FAILED  
diehard_count_1s_byt|   0|    256000|     100|0.68858185|  PASSED  
 diehard_parking_lot|   0|     12000|     100|0.99650356|   WEAK   
    diehard_2dsphere|   2|      8000|     100|0.07461599|  PASSED  
    diehard_3dsphere|   3|      4000|     100|0.55283492|  PASSED  
     diehard_squeeze|   0|    100000|     100|0.00000000|  FAILED  
        diehard_sums|   0|       100|     100|0.76714923|  PASSED  
        diehard_runs|   0|    100000|     100|0.00793089|  PASSED  
        diehard_runs|   0|    100000|     100|0.22090600|  PASSED  
       diehard_craps|   0|    200000|     100|0.00000000|  FAILED  
       diehard_craps|   0|    200000|     100|0.00748931|  PASSED  
 marsaglia_tsang_gcd|   0|  10000000|     100|0.00000000|  FAILED  
 marsaglia_tsang_gcd|   0|  10000000|     100|0.00000000|  FAILED  
         sts_monobit|   1|    100000|     100|0.00000000|  FAILED  
            sts_runs|   2|    100000|     100|0.00000000|  FAILED  
          sts_serial|   1|    100000|     100|0.00000000|  FAILED  
          sts_serial|   2|    100000|     100|0.00000000|  FAILED  
          sts_serial|   3|    100000|     100|0.00000000|  FAILED  
          sts_serial|   3|    100000|     100|0.00000000|  FAILED  
          sts_serial|   4|    100000|     100|0.00000000|  FAILED  
          sts_serial|   4|    100000|     100|0.00000000|  FAILED  
          sts_serial|   5|    100000|     100|0.00000000|  FAILED  
          sts_serial|   5|    100000|     100|0.00000000|  FAILED  
          sts_serial|   6|    100000|     100|0.00000000|  FAILED  
          sts_serial|   6|    100000|     100|0.00000000|  FAILED  
          sts_serial|   7|    100000|     100|0.00000000|  FAILED  
          sts_serial|   7|    100000|     100|0.00000000|  FAILED  
          sts_serial|   8|    100000|     100|0.00000000|  FAILED  
          sts_serial|   8|    100000|     100|0.00000000|  FAILED  
          sts_serial|   9|    100000|     100|0.00000000|  FAILED  
          sts_serial|   9|    100000|     100|0.00000000|  FAILED  
          sts_serial|  10|    100000|     100|0.00000000|  FAILED  
          sts_serial|  10|    100000|     100|0.00000000|  FAILED  
          sts_serial|  11|    100000|     100|0.00000000|  FAILED  
          sts_serial|  11|    100000|     100|0.00000000|  FAILED  
          sts_serial|  12|    100000|     100|0.00000000|  FAILED  
          sts_serial|  12|    100000|     100|0.00000000|  FAILED  
          sts_serial|  13|    100000|     100|0.00000000|  FAILED  
          sts_serial|  13|    100000|     100|0.00000000|  FAILED  
          sts_serial|  14|    100000|     100|0.00000000|  FAILED  
          sts_serial|  14|    100000|     100|0.00000000|  FAILED  
          sts_serial|  15|    100000|     100|0.00000000|  FAILED  
          sts_serial|  15|    100000|     100|0.00000000|  FAILED  
          sts_serial|  16|    100000|     100|0.00000000|  FAILED  
          sts_serial|  16|    100000|     100|0.00000000|  FAILED  
         rgb_bitdist|   1|    100000|     100|0.00000000|  FAILED  
         rgb_bitdist|   2|    100000|     100|0.00000000|  FAILED  
         rgb_bitdist|   3|    100000|     100|0.00001552|   WEAK   
         rgb_bitdist|   4|    100000|     100|0.00000000|  FAILED  
         rgb_bitdist|   5|    100000|     100|0.00000000|  FAILED  
         rgb_bitdist|   6|    100000|     100|0.00000806|   WEAK   
         rgb_bitdist|   7|    100000|     100|0.07378409|  PASSED  
         rgb_bitdist|   8|    100000|     100|0.23721258|  PASSED  
         rgb_bitdist|   9|    100000|     100|0.01635462|  PASSED  
# The file file_input_raw was rewound 1 times
         rgb_bitdist|  10|    100000|     100|0.41476870|  PASSED  
# The file file_input_raw was rewound 1 times
         rgb_bitdist|  11|    100000|     100|0.20763795|  PASSED  
# The file file_input_raw was rewound 1 times
         rgb_bitdist|  12|    100000|     100|0.03961637|  PASSED  
# The file file_input_raw was rewound 1 times
rgb_minimum_distance|   2|     10000|    1000|0.00000000|  FAILED  
# The file file_input_raw was rewound 1 times
rgb_minimum_distance|   3|     10000|    1000|0.84189770|  PASSED  
# The file file_input_raw was rewound 1 times
rgb_minimum_distance|   4|     10000|    1000|0.07201586|  PASSED  
# The file file_input_raw was rewound 1 times
rgb_minimum_distance|   5|     10000|    1000|0.04425926|  PASSED  
# The file file_input_raw was rewound 1 times
    rgb_permutations|   2|    100000|     100|0.91069440|  PASSED  
# The file file_input_raw was rewound 1 times
    rgb_permutations|   3|    100000|     100|0.44047246|  PASSED  
# The file file_input_raw was rewound 1 times
    rgb_permutations|   4|    100000|     100|0.72389934|  PASSED  
# The file file_input_raw was rewound 1 times
    rgb_permutations|   5|    100000|     100|0.21941374|  PASSED  
# The file file_input_raw was rewound 1 times
      rgb_lagged_sum|   0|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 1 times
      rgb_lagged_sum|   1|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 1 times
      rgb_lagged_sum|   2|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 1 times
      rgb_lagged_sum|   3|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 1 times
      rgb_lagged_sum|   4|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 1 times
      rgb_lagged_sum|   5|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 1 times
      rgb_lagged_sum|   6|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 2 times
      rgb_lagged_sum|   7|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 2 times
      rgb_lagged_sum|   8|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 2 times
      rgb_lagged_sum|   9|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 2 times
      rgb_lagged_sum|  10|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 3 times
      rgb_lagged_sum|  11|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 3 times
      rgb_lagged_sum|  12|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 3 times
      rgb_lagged_sum|  13|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 3 times
      rgb_lagged_sum|  14|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 4 times
      rgb_lagged_sum|  15|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 4 times
      rgb_lagged_sum|  16|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 5 times
      rgb_lagged_sum|  17|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 5 times
      rgb_lagged_sum|  18|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 6 times
      rgb_lagged_sum|  19|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 6 times
      rgb_lagged_sum|  20|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 7 times
      rgb_lagged_sum|  21|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 7 times
      rgb_lagged_sum|  22|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 8 times
      rgb_lagged_sum|  23|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 8 times
      rgb_lagged_sum|  24|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 9 times
      rgb_lagged_sum|  25|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 9 times
      rgb_lagged_sum|  26|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 10 times
      rgb_lagged_sum|  27|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 11 times
      rgb_lagged_sum|  28|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 12 times
      rgb_lagged_sum|  29|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 12 times
      rgb_lagged_sum|  30|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 13 times
      rgb_lagged_sum|  31|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 14 times
      rgb_lagged_sum|  32|   1000000|     100|0.00000000|  FAILED  
# The file file_input_raw was rewound 14 times
     rgb_kstest_test|   0|     10000|    1000|0.00000000|  FAILED  
# The file file_input_raw was rewound 14 times
     dab_bytedistrib|   0|  51200000|       1|0.00000000|  FAILED  
# The file file_input_raw was rewound 14 times
             dab_dct| 256|     50000|       1|0.59573224|  PASSED  
# The file file_input_raw was rewound 14 times
Preparing to run test 207.  ntuple = 0
        dab_filltree|  32|  15000000|       1|0.60303646|  PASSED  
        dab_filltree|  32|  15000000|       1|0.54889797|  PASSED  
# The file file_input_raw was rewound 14 times
Preparing to run test 208.  ntuple = 0
       dab_filltree2|   0|   5000000|       1|0.74919512|  PASSED  
       dab_filltree2|   1|   5000000|       1|0.00000000|  FAILED  
# The file file_input_raw was rewound 14 times
Preparing to run test 209.  ntuple = 0
        dab_monobit2|  12|  65000000|       1|1.00000000|  FAILED

Clearly some of these tests failed due to the file being rewound due to not having enough RNG, however the majority of these tests should be passing for an RNG source thats used for cryptographic operations. Has Espressif ran these tests in-house? Thanks!

Update: The source of these failing results is probably because I was outputing a raw binary stream over UART, and (by default) esp-idf replaces "\n" with "\r\n".
Last edited by bnp117 on Fri Oct 11, 2019 3:38 pm, edited 1 time in total.
Top
bnp117
Posts: 17
Joined: Fri Jun 15, 2018 3:08 am

Re: RNG Quality (Dieharder Test Results Failing)

Postby bnp117 » Thu Oct 10, 2019 3:09 pm

bump
Top
Markus Becker
Posts: 22
Joined: Fri Mar 02, 2018 3:24 pm

Re: RNG Quality (Dieharder Test Results Failing)

Postby Markus Becker » Thu Oct 10, 2019 3:57 pm

Hi bnp117,
well, I'm working on reproducing your test, just it takes time to collect data... 8-)
Having said this, as not being an expert, I'm not completely sure about the relevance of dieharder to judge the quality of a TRNG.
I also see tests failing when doing `dieharder -g 501 -a` on a Ubuntu box, investigating...
Best
Markus
Top
bnp117
Posts: 17
Joined: Fri Jun 15, 2018 3:08 am

Re: RNG Quality (Dieharder Test Results Failing)

Postby bnp117 » Thu Oct 10, 2019 7:45 pm

Hi Markus,

Thanks for taking a look! And dieharder is the defacto RNG testing suite that is very sensitive to non-rng biases from an RNG source. Failing or having weak scores in the tests indicates that your entropy is actually less than you may expect from a TRNG.
Top
ESP_Angus
Posts: 2344
Joined: Sun May 08, 2016 4:11 am

Re: RNG Quality (Dieharder Test Results Failing)

Postby ESP_Angus » Thu Oct 10, 2019 11:33 pm

Hi bnp117,

We also ran the dieharder suite on ESP32 output and it passed.

Is it possible you could please post the project that was used for the test & to capture the data to the PC? You can send it to me privately if you don't wish to share it publicly.

Angus
Top
bnp117
Posts: 17
Joined: Fri Jun 15, 2018 3:08 am

Re: RNG Quality (Dieharder Test Results Failing)

Postby bnp117 » Fri Oct 11, 2019 4:05 am

It's part of a much bigger project:

https://github.com/joltwallet/jolt_wallet

The important parts are:

1. I have a function that generates n bytes of data by calling `esp_random()` a bunch of times
2. I have a console command that spews out those bytes to stdout
Top
ESP_Angus
Posts: 2344
Joined: Sun May 08, 2016 4:11 am

Re: RNG Quality (Dieharder Test Results Failing)

Postby ESP_Angus » Fri Oct 11, 2019 4:30 am

Hi bnp117,

I see, thanks for the links. Are you outputting the data in hex or raw binary form?

If raw binary, check this kconfig item, the default stdout setting will replace any \n with \r\n and this pattern is (rightly) enough to fail dieharder. (I know this from having debugged it once before.)

If that's not it, can you share any details about how you're capturing the output from the UART and processing it into dieharder?

Angus
Top
Markus Becker
Posts: 22
Joined: Fri Mar 02, 2018 3:24 pm

Re: RNG Quality (Dieharder Test Results Failing)

Postby Markus Becker » Fri Oct 11, 2019 11:35 am

Hi bnp117,
done, here the result from my testing:

Code: Select all

Starting test output
#=============================================================================#
#            dieharder version 3.31.1 Copyright 2003 Robert G. Brown          #
#=============================================================================#
   rng_name    |rands/second|   Seed   |
stdin_input_raw|  2.67e+07  | 221663847|
#=============================================================================#
        test_name   |ntup| tsamples |psamples|  p-value |Assessment
#=============================================================================#
   diehard_birthdays|   0|       100|     100|0.92147329|  PASSED  
      diehard_operm5|   0|   1000000|     100|0.92257746|  PASSED  
  diehard_rank_32x32|   0|     40000|     100|0.63096925|  PASSED  
    diehard_rank_6x8|   0|    100000|     100|0.18101713|  PASSED  
   diehard_bitstream|   0|   2097152|     100|0.75752915|  PASSED  
        diehard_opso|   0|   2097152|     100|0.48664437|  PASSED  
        diehard_oqso|   0|   2097152|     100|0.93119132|  PASSED  
         diehard_dna|   0|   2097152|     100|0.50497207|  PASSED  
diehard_count_1s_str|   0|    256000|     100|0.62077238|  PASSED  
diehard_count_1s_byt|   0|    256000|     100|0.23561322|  PASSED  
 diehard_parking_lot|   0|     12000|     100|0.49824922|  PASSED  
    diehard_2dsphere|   2|      8000|     100|0.78678132|  PASSED  
    diehard_3dsphere|   3|      4000|     100|0.71728932|  PASSED  
     diehard_squeeze|   0|    100000|     100|0.86010898|  PASSED  
        diehard_sums|   0|       100|     100|0.00131877|   WEAK   
        diehard_runs|   0|    100000|     100|0.67455792|  PASSED  
        diehard_runs|   0|    100000|     100|0.72982775|  PASSED  
       diehard_craps|   0|    200000|     100|0.48955723|  PASSED  
       diehard_craps|   0|    200000|     100|0.61364953|  PASSED  
 marsaglia_tsang_gcd|   0|  10000000|     100|0.89758017|  PASSED  
 marsaglia_tsang_gcd|   0|  10000000|     100|0.85542069|  PASSED  
         sts_monobit|   1|    100000|     100|0.99732156|   WEAK   
            sts_runs|   2|    100000|     100|0.95003812|  PASSED  
          sts_serial|   1|    100000|     100|0.89008866|  PASSED  
          sts_serial|   2|    100000|     100|0.46699321|  PASSED  
          sts_serial|   3|    100000|     100|0.28751636|  PASSED  
          sts_serial|   3|    100000|     100|0.56814502|  PASSED  
          sts_serial|   4|    100000|     100|0.94989811|  PASSED  
          sts_serial|   4|    100000|     100|0.88074085|  PASSED  
          sts_serial|   5|    100000|     100|0.32720567|  PASSED  
          sts_serial|   5|    100000|     100|0.37386004|  PASSED  
          sts_serial|   6|    100000|     100|0.61263842|  PASSED  
          sts_serial|   6|    100000|     100|0.49172679|  PASSED  
          sts_serial|   7|    100000|     100|0.90528489|  PASSED  
          sts_serial|   7|    100000|     100|0.77508049|  PASSED  
          sts_serial|   8|    100000|     100|0.48042346|  PASSED  
          sts_serial|   8|    100000|     100|0.58758168|  PASSED  
          sts_serial|   9|    100000|     100|0.75644157|  PASSED  
          sts_serial|   9|    100000|     100|0.40341920|  PASSED  
          sts_serial|  10|    100000|     100|0.93276338|  PASSED  
          sts_serial|  10|    100000|     100|0.09885333|  PASSED  
          sts_serial|  11|    100000|     100|0.89169241|  PASSED  
          sts_serial|  11|    100000|     100|0.43275009|  PASSED  
          sts_serial|  12|    100000|     100|0.95265163|  PASSED  
          sts_serial|  12|    100000|     100|0.99641910|   WEAK   
          sts_serial|  13|    100000|     100|0.36068052|  PASSED  
          sts_serial|  13|    100000|     100|0.05408159|  PASSED  
          sts_serial|  14|    100000|     100|0.94137897|  PASSED  
          sts_serial|  14|    100000|     100|0.55580789|  PASSED  
          sts_serial|  15|    100000|     100|0.41986920|  PASSED  
          sts_serial|  15|    100000|     100|0.26100150|  PASSED  
          sts_serial|  16|    100000|     100|0.96869455|  PASSED  
          sts_serial|  16|    100000|     100|0.17584534|  PASSED  
         rgb_bitdist|   1|    100000|     100|0.65935025|  PASSED  
         rgb_bitdist|   2|    100000|     100|0.45831683|  PASSED  
         rgb_bitdist|   3|    100000|     100|0.90111899|  PASSED  
         rgb_bitdist|   4|    100000|     100|0.21020947|  PASSED  
         rgb_bitdist|   5|    100000|     100|0.52762477|  PASSED  
         rgb_bitdist|   6|    100000|     100|0.89747369|  PASSED  
         rgb_bitdist|   7|    100000|     100|0.73325909|  PASSED  
         rgb_bitdist|   8|    100000|     100|0.64996160|  PASSED  
         rgb_bitdist|   9|    100000|     100|0.57048929|  PASSED  
         rgb_bitdist|  10|    100000|     100|0.41585597|  PASSED  
         rgb_bitdist|  11|    100000|     100|0.00516434|  PASSED  
         rgb_bitdist|  12|    100000|     100|0.57490767|  PASSED  
rgb_minimum_distance|   2|     10000|    1000|0.87914993|  PASSED  
rgb_minimum_distance|   3|     10000|    1000|0.03420639|  PASSED  
rgb_minimum_distance|   4|     10000|    1000|0.70899908|  PASSED  
rgb_minimum_distance|   5|     10000|    1000|0.55797566|  PASSED  
    rgb_permutations|   2|    100000|     100|0.82406249|  PASSED  
    rgb_permutations|   3|    100000|     100|0.72515871|  PASSED  
    rgb_permutations|   4|    100000|     100|0.44187170|  PASSED  
    rgb_permutations|   5|    100000|     100|0.78557362|  PASSED  
      rgb_lagged_sum|   0|   1000000|     100|0.52784546|  PASSED  
      rgb_lagged_sum|   1|   1000000|     100|0.94970534|  PASSED  
      rgb_lagged_sum|   2|   1000000|     100|0.72453669|  PASSED  
      rgb_lagged_sum|   3|   1000000|     100|0.15810963|  PASSED  
Out of data - reset
      rgb_lagged_sum|   4|   1000000|     100|0.35424029|  PASSED  
      rgb_lagged_sum|   5|   1000000|     100|0.77398353|  PASSED  
      rgb_lagged_sum|   6|   1000000|     100|0.30475135|  PASSED  
      rgb_lagged_sum|   7|   1000000|     100|0.71551348|  PASSED  
      rgb_lagged_sum|   8|   1000000|     100|0.31995168|  PASSED  
      rgb_lagged_sum|   9|   1000000|     100|0.99960506|   WEAK   
      rgb_lagged_sum|  10|   1000000|     100|0.47909163|  PASSED  
Out of data - reset
      rgb_lagged_sum|  11|   1000000|     100|0.64565979|  PASSED  
      rgb_lagged_sum|  12|   1000000|     100|0.73055225|  PASSED  
      rgb_lagged_sum|  13|   1000000|     100|0.13378502|  PASSED  
      rgb_lagged_sum|  14|   1000000|     100|0.99952653|   WEAK   
      rgb_lagged_sum|  15|   1000000|     100|0.93192984|  PASSED  
Out of data - reset
      rgb_lagged_sum|  16|   1000000|     100|0.77683593|  PASSED  
      rgb_lagged_sum|  17|   1000000|     100|0.21538976|  PASSED  
      rgb_lagged_sum|  18|   1000000|     100|0.23699995|  PASSED  
Out of data - reset
      rgb_lagged_sum|  19|   1000000|     100|0.78095094|  PASSED  
      rgb_lagged_sum|  20|   1000000|     100|0.33662429|  PASSED  
      rgb_lagged_sum|  21|   1000000|     100|0.89729160|  PASSED  
Out of data - reset
      rgb_lagged_sum|  22|   1000000|     100|0.63386356|  PASSED  
      rgb_lagged_sum|  23|   1000000|     100|0.86128842|  PASSED  
      rgb_lagged_sum|  24|   1000000|     100|0.39325756|  PASSED  
Out of data - reset
      rgb_lagged_sum|  25|   1000000|     100|0.89360955|  PASSED  
      rgb_lagged_sum|  26|   1000000|     100|0.56952133|  PASSED  
Out of data - reset
      rgb_lagged_sum|  27|   1000000|     100|0.31729725|  PASSED  
      rgb_lagged_sum|  28|   1000000|     100|0.27236571|  PASSED  
Out of data - reset
      rgb_lagged_sum|  29|   1000000|     100|0.55375343|  PASSED  
      rgb_lagged_sum|  30|   1000000|     100|0.07479765|  PASSED  
Out of data - reset
      rgb_lagged_sum|  31|   1000000|     100|0.74924472|  PASSED  
      rgb_lagged_sum|  32|   1000000|     100|0.43174817|  PASSED  
     rgb_kstest_test|   0|     10000|    1000|0.00094866|   WEAK   
     dab_bytedistrib|   0|  51200000|       1|0.50310042|  PASSED  
             dab_dct| 256|     50000|       1|0.50775262|  PASSED  
Preparing to run test 207.  ntuple = 0
        dab_filltree|  32|  15000000|       1|0.08716861|  PASSED  
        dab_filltree|  32|  15000000|       1|0.22849843|  PASSED  
Preparing to run test 208.  ntuple = 0
       dab_filltree2|   0|   5000000|       1|0.84691425|  PASSED  
       dab_filltree2|   1|   5000000|       1|0.31729935|  PASSED  
Preparing to run test 209.  ntuple = 0
        dab_monobit2|  12|  65000000|       1|0.70712340|  PASSED  
Out of data - reset
Out of data - reset
Out of data - reset
...
The result of my testing differs quite substantial from yours.

What I did:
The test ran on a custom board, based on a WROOM32 module. The device has some IO including SD card.
Bt is on, WiFi is off.
I chose to add the test to an existing project, here is the proc used to capture the data:

Code: Select all

static void task_test_rng_sd(){
	FILE * file = NULL;
    char filename[ 80];
    filename[ 0] = 0;
    uint32_t file_nr = 0;
    uint32_t ec;
    uint32_t* rand_buffer = NULL;
    uint32_t bytes_written;
    uint32_t FILE_SIZE = 1024*1024*1000;
    uint32_t last_log = 0;
    uint32_t last_bytes_written = 0;

    rand_buffer = malloc( 512);
    if (rand_buffer == NULL) goto err_exit;

    ESP_LOGE( TAG, "RAND test begin...");
    vTaskDelay( 2000 / portTICK_PERIOD_MS);
	while (sdcard_init() != SDCARD_MOUNTED){
		sdcard_done();
	    ESP_LOGE( TAG, "RAND test. Plug in SD!");
		vTaskDelay( 10000 / portTICK_PERIOD_MS);
	}
    while( 1){
    	if (!file){
    	    sprintf( filename, "/sdcard/%d.raw", file_nr);
    	    file = fopen(filename, "w+");
    	    if (!file) {
    	    	ESP_LOGE(TAG, "Error open %s",filename);
    			ec = 1;
    			goto err_exit;
    	    }
    	    bytes_written = 0;
    	    last_log = tbox_millis();
    	    last_bytes_written = 0;
    	    ESP_LOGI( TAG, "writing %s", filename);
    	}
    	for (uint16_t i = 0;i<128;i++){
    		rand_buffer[ i] = esp_random();
    	}
    	size_t write_count = fwrite(rand_buffer, 512, 1, file);
    	if (write_count!=1){
    		ec = 2;
    		goto err_exit;
    	}
    	bytes_written+=512;
    	if (bytes_written>=FILE_SIZE){
    		fclose( file);
    		file=NULL;
    		file_nr++;
    	}
    	uint32_t t1 = tbox_millis()-last_log;
    	if (t1>10000){
    		uint32_t w = bytes_written - last_bytes_written;
    		last_bytes_written =  bytes_written;
    		uint32_t ws = w / 1024 * 1000 / t1;
    	    ESP_LOGI( TAG, "written %d at %dkb/s", bytes_written, ws);
    	    last_log = tbox_millis();
    	}
    	vTaskDelay( 0);
    }

err_exit:
	if (file) fclose( file);
	file = NULL;
	free( rand_buffer);
	ESP_LOGE( TAG, "RAND test finish code %d", ec);
}
There are some dependencies not included here, I hope you can understand the purpose of the code. Also, pls don't judge code quality, this is just a quick test.
The procedure should wait for a SD card, then repeat to write files of 1gb of esp_random() output to files until an error (disk full, SD removed, ...) comes up.
Writing to the SD card was 200kb/sec to 355kb/sec, resulting in 24*1gb data in about a day (about 50 minutes to collect 1gb).

Then, after moving the data to an Ubuntu box, I used the naive script

Code: Select all

#!/bin/bash 
echo "Starting test output" >&2
while [  1 ]; do
  cat /media/nvme/rng_test/0.RAW
  cat /media/nvme/rng_test/1.RAW
  cat /media/nvme/rng_test/2.RAW
  cat /media/nvme/rng_test/3.RAW
  cat /media/nvme/rng_test/4.RAW
  cat /media/nvme/rng_test/5.RAW
  cat /media/nvme/rng_test/6.RAW
  cat /media/nvme/rng_test/7.RAW
  cat /media/nvme/rng_test/8.RAW
  cat /media/nvme/rng_test/9.RAW
  cat /media/nvme/rng_test/10.RAW
  cat /media/nvme/rng_test/11.RAW
  cat /media/nvme/rng_test/12.RAW
  cat /media/nvme/rng_test/13.RAW
  cat /media/nvme/rng_test/14.RAW
  cat /media/nvme/rng_test/15.RAW
  cat /media/nvme/rng_test/16.RAW
  cat /media/nvme/rng_test/17.RAW
  cat /media/nvme/rng_test/18.RAW
  cat /media/nvme/rng_test/19.RAW
  cat /media/nvme/rng_test/20.RAW
  cat /media/nvme/rng_test/21.RAW
  cat /media/nvme/rng_test/22.RAW
  cat /media/nvme/rng_test/23.RAW
  echo "Out of data - reset" >&2
done
to feed dieharder like this

Code: Select all

esp32@mb-server:~$ (./rng.sh | dieharder -g 200 -a) > rng_res.txt 2>&1
As a side note, I think passing this kind of test should not be overestimated in terms of "how good/useful/secure/..." is the ESP32 TRNG.

Best
Markus
Top
bnp117
Posts: 17
Joined: Fri Jun 15, 2018 3:08 am

Re: RNG Quality (Dieharder Test Results Failing)

Postby bnp117 » Fri Oct 11, 2019 3:37 pm

@ESP_Angus I was outputting raw binary; you're absolutely correct these failures are probably causes by the newlinereplacement. Thank you so much! I'll be running another test, but all the evidence suggests that this was the bug.

@Markus Becker thank you so much for independently programming and testing this RNG experiment. Dieharder isn't meant to be an end-all be-all for RNG quality, but failing some dieharder tests should make you either reconsider your RNG source or find a bug somewhere along the path.
Top
Post Reply

Return to “General Discussion”

Who is online

Users browsing this forum: Google [Bot] and 56 guests