Bug in 1.94 with 38k
#1
Some people have run into problems with CometChat hanging when using WiModem at 38k.  I did some troubleshooting this weekend and think I've root-caused it to the WiModem.

When using 38K, sequences of characters longer than 128 sent in close proximity seem to be getting truncated.  This even occurs without screen-blanking. enabled, which means that communication only occurs while the scanline is in the vertical border. 

If I had to speculate (and I know... I don't, but I can't help it... :p)  ...  I'd say there is a 128-byte buffer for receiving from the C64, and the packetization parameters are not tuned well, maybe a watermark needs to be reduced.  I'm kinda surprised that it happens with screen-blanking disabled, because in that mode only about 11 bytes may be transferred per VIC frame, with over 13 milliseconds between each group of 11.

I've only tested this with a program that generates one chat-oriented burst of bytes at a time, and not with the SAVE command, but I'm 99.9% sure that saves will be broken too.  I'll try to test that and post the results.

Let me know if you'd like me to share my test code.
// Agent Friday
Reply
#2
There is a 128 byte internal hardware buffer for both send and receiver (256 total).  This is a limitation of the WiFi module (ESP8266) itself.  What do you mean by "watermark"?  Does your driver recognize when CTS is asserted to stop sending of data from the C64 to the modem?  That occurs when the buffer is full.
Reply
#3
Yep, save doesn't work right in 38k either. I was able to save a 116-byte program just fine. But got a hang when trying to save a 153-byte program. I broke out and sent 26 bytes of garbage, which resulted in the "00 - SAVE OK" response coming back from commodoreserver.

Analysis: Saving a 153-byte file will result in 155 bytes of data (to include load address) being sent after getting the go-ahead prompt back from CS. The fact that 26 bytes of data were required to complete the save must mean that 155-26 = 129 bytes of data got through before bytes began dropping. Interesting...
Reply
#4
(06-12-2017, 03:19 PM)admin Wrote: There is a 128 byte internal hardware buffer for both send and receiver (256 total).  This is a limitation of the WiFi module (ESP8266) itself.  What do you mean by "watermark"?
I think the proper term is high-water mark... Just a threshold that activates packetization.  If non-zero time is required to make room in the buffer, then the process would need to be started soon enough that the buffer pressure is relieved before it overflows.


(06-12-2017, 03:19 PM)admin Wrote: Does your driver recognize when CTS is asserted to stop sending of data from the C64 to the modem?  That occurs when the buffer is full.
Yes, the 38k driver is fully responsive to the CTS line, and if used properly should be able to throttle outgoing data indefinitely without loss.
Reply
#5
Maybe what is going on is that it is being asserted too late.  I will have to check that.  There can be a long period of time where no data can be sent from the C64 (like when receive data is being sent to the C64).

The ESP8266 automatically handles the output buffer and packetization.  The only thing you can't do is send too much data to it, so CTS has to be used to prevent that.  Since the issue only occurs when sending data from the C64 to the WiModem, I will focus on that part.  I know that the receiving works fine.  I can LOAD a file over any size from Commodore server.  However, I do double-buffer the WiModem->C64.  Maybe I need to do the same thing for the C64->WiModem.
Reply
#6
If you do forward the CTS signal to the C64, it checks it before going into each start-bit, so at most 1 byte would be in "limbo". Maybe let's start by confirming that CTS gets asserted when it should? If for some reason the driver's not obeying it, I'll fix that. Ideally tho... if the need to throttle could be eliminated, maximum throughput could be achieved.

It'd be good to have a series of low-level tests that would prove full functionality on both ends, and also that demonstrate what the real-world throughput is. I could write the test suite if you wanted to work with me on the specs.
Reply
#7
There are people sending and receiving up to 1MB/s using the ESP8266 using specific setups, but it seems a few hundred K per second is the norm.

So, sure I would be happy to work this out with you.
Reply
#8
Well, I am pretty sure that I found the problem. I am fetching [receive buff,len] worth of data and passing that to the ESP8266 module. However, when len>128 the packet is not broken up correctly. I will work on this.
Reply
#9
Well, after further investigating the problem is not with the internet side.  The ESP8266 can handle up to 2920 bytes at one time with the current libraries, and the new release will extend that to 4K.  There is still a 128 byte receive buffer, so perhaps that is over-flowing before there is a chance to service it.  That is odd though, there really is nothing going on other than polling for data.
Reply
#10
I verified that the receive buffer is not overflowing, but I am going to put a check for the largest block of data that is passed from the computer to see where the threshold currently is.
Reply




Users browsing this thread: 1 Guest(s)