I might have found what caused the problem stated here.
The scanner sketch uses the same code to process both the advertising packets and scan response packets. The MSD filter is universal and applies to all incoming packets. The first thing is to make sure both packets are of a common structure otherwise we should not turn on the filter.
After enabling active scanning on the scanner, the BLE stack will wait for the scan response after seeing the scan request. Referring to the current profile of the peripheral, the transmission of advertising and scan response packets takes place within a very short time. If the scanner does not dump all the buffer contents immediately upon receipt and resumes scanning, the buffer will easily be overwritten by next incoming packet.
If the scanner works properly one shall see the following at the serial output. Look at the timing, the scan response is captured within the next milli second after the advertising packet.
If we slow down the scanner by introducing some meaningless tasks, the scan response will vanish since it is always overwritten by the next advertising packet.
/* Check for Manufacturer Specific Data */
len = Bluefruit.Scanner.parseReportByType(report, BLE_GAP_AD_TYPE_MANUFACTURER_SPECIFIC_DATA, buffer, sizeof(buffer));
Serial.printf("%14s ", “MAN SPEC DATA”);
Serial.printBuffer(buffer, len, ‘-’);
memset(buffer, 0, sizeof(buffer));
for (int i = 0; i <= 3; i++)
Serial.println("What the Hell??? ");
// For Softdevice v6: after received a report, scanner will be paused
// We need to call Scanner resume() to continue scanning
The serial output will become like this:
The faster the peripheral is broadcasting, the lesser time we got to process the data.
Finally, to explain why the scanner could sometimes (1~5%) pick up the scan response with my earlier sketch. Well, it did not always capture the response initiated by itself. It sometimes should have captured the response initiated by other BLE devices in the vicinity.
EDIT: the reason of being unable to print out SR packets was due to callback routine being held up too long to process the ADV packet and missed the callback for SR packet which arrived within half a milli second after the previous ADV packet. MS has found the proper solution, please scroll down to see his sketch.