Posts: 2,839
Threads: 148
Joined: Nov 2013
We have made the flux image format public information so that emulation authors can finally take advantage of low level data that is easy to read and use.
The flux image format consists of a header containing offsets to tracks. Each track contains a header with the information about the track data, including bit cell count, rotational speed, and offset to the actual flux data. The image format was made so that corrupted images could be recovered, losing only the track(s) of data where the corruption occurred, not all of the data. Because offsets are used for the data, it is possible to make very small images where writing is not required (a write-protected disk). You can also have a large padded space so that the track can change sizes for writing support.
If you have any questions about the data structure, please ask here so that everyone can learn.
We are willing to help emulation authors implement this format. This format is being implemented into the FPGA Arcade for low-level disk access so it will be possible to use copy protected disk images that will load just like the original!
The updated image file format specification is located here:
https://www.cbmstuff.com/downloads/scp/s..._specs.txt
Posts: 27
Threads: 0
Joined: Dec 2013
I am considering to support .scp in some of my tools for Atari.
Would you have some .scp file of Atari diskette for test purpose?
Even basic FD will do.
Posts: 27
Threads: 0
Joined: Dec 2013
12-19-2013, 09:54 AM
(This post was last modified: 12-19-2013, 09:58 AM by DrCoolZic.)
I have read the scp image spec and it look nice and simple
Just to make sure I got it right:
- the actual data for transitions flux recording are located by areas pointed by DATA offset right?
- Each time the HW detects a transition this will end up as one entry in the "Data segment". For example if we have the following transition sequence: 4µs, 6µs, 8µs etc ... this will be stored as 3 bytes with value 160, 240, 320 etc...
- One critical point: I do not see any information about the position of the transitions in respect to the index. In most cases this is not important but it can be critical in certain situation like no flux area where you can have flux transitions spaced by SEVERAL MILISECONDS! In that case it is important to know where the index is located with regard to the data
Data ______//^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\________
Index _______________________________|^^^^|____________________________
Here is an example of a no flux area (NFA) located just above the index pulse
As you can see the NFA starts around 198.6 ms for a time of 2993 µs and therefore end up at the begining of the track (as the disk spin )
Posts: 2,839
Threads: 148
Joined: Nov 2013
Your assumptions are correct. However, what you call "no flux areas" are actually really long flux transitions, and yes, they can be very long or very short and in groups as well. For example, it's not uncommon to see multiple flux transitions that are 22,000us to 27,000us on weakbit protected disks. The PC disk controllers do return a time duration, even if it is invalid so there really is no such thing as a "no flux area". That area will return random bit cell times each time read, but these all add up to the correct time. So, the bit cell position in reference to the index mark is the total time accumulated up to the point of interest. You can see in my visualizer data there is a total bit length in bytes and a total time of all of the bit cells added together. That time should match the index to index time within a few microseconds, depending on drive speed variations.
What you show in your data is not a no flux area, otherwise you would have no mid/high flux patterns shown like you do. Even if you turn on the erase head and output a 0 or a 1 to the disk, it is going to return flux transitions when the disk is read. Will they be correct? No, but they will be there as random transitions due to how the drive controllers function. Of course, this is somewhat drive controller specific as they all do slightly different things. Some drives return a long series of super short transitions, some return super long transitions, and some return about 25% more than longest possible bit cell duration.
I just noticed that the spec info attached to the original post is very outdated and changed some. Please see the new spec (now as a download).
Posts: 27
Threads: 0
Joined: Dec 2013
I think there is probably a problem of terminology or a misunderstanding.
Below is my explanation on No Flux Area (or whatever you call it) and how it is detected in a game like Turrican (this is from one of my doc)
Code: There has been a lot of debate around the so called No Flux Area on disk. This is a protection's mechanism used on some floppies that results in absolutely no flux transitions coming from the drive read circuitry for a relatively long period of time (usually several milliseconds).
For some times it has been thought that this was obtained by doing a so called "strong erasure" on areas of a disk. However this would be very difficult to create and would not produce the wanted effect:
For one this can’t be done with the normal recording head/circuitry of a floppy drive and therefore it would require to use modified drives.
Secondly if such areas, with no magnetic flux transitions, existed on the floppy disk it would cause the ACG of the read chain to be set to its maximum amplification value and this would result in reading random flux transitions which is not the case.
Bit-shift occurs on any NRZ recorded medium as a normal consequence from read/write head operation. Data are written when the read/write head generates a flux change in the gap of the head, which causes a change in magnetization of the medium oxide. In reading, a current is induced into the read/write head when a flux transition on the medium is encountered. The current change is not instantaneous, since it takes a finite time to build up to peak and then to return to zero. If flux transitions are close together, the current buildup after one flux transition then declines, but it does not have time to reach zero before the second transition begins. Consequently current pulses are summed by the read/write head, which causes the peaks to be shifted.
A No Flux Area is created by writing a large number of flux transitions close enough (i.e. at a relatively high frequency). This will result in having the read current never returning to zero and consequently this will result as no data pulse generated on the read channel. Note that in this case the ACG is not set to a maximum amplification as the input circuitry receives flux transitions even if no data is coming out of the read channel.
How is-it possible to check an NFA with a standard WD1772 Floppy disk controller? Normally the WD1772 FDC can only read the data bits. Therefore a sector with NFA is read as a sector filled of bytes 0x00 but it is normally not possible to check that the clock bits are also bytes 0x00. To be able to check the clock bits the NFA protection uses an interesting trick. Another sector is written within the sector (SWS) that contains the NFA and this sector contains 3 sync marks shifted by one bit cell. Therefore when you read the data for this second sector you are actually reading the clock data from the first one!
I have zoomed on a NFA located in range 89-94 ms.
Of course you have to remember that normally a 4.3ms flux transition is not possible and it is not either an unformated area that would generate noise ...
see also http://bitsavers.trailing-edge.com/pdf/m...ystems.pdf
and
http://info-coach.fr/atari/hardware/FD-Hard.php#writing
and
http://info-coach.fr/atari/hardware/fd-hard/AN-413.pdf
Posts: 2,839
Threads: 148
Joined: Nov 2013
Thanks for the info. Yes, I believe there is some terminology difference here. This is what we called strongbits for the Amiga, and I am pretty sure that Turrican used this protection for the Amiga as well. We could duplicate it without hardware though because the Amiga's disk controller was so versatile. The more I read about the Atari ST stuff, the more I realize why we sold so many SuperCard Ami units to the Atari ST clubs. They used it with an Amiga 500 to duplicate all of the ST disks. Until the SuperCard Pro project, I had never seen a decoded MFM track (PC or Atari ST). I have always looked at everything as raw MFM (or flux since SuperCard Pro). I can see why you guys had your hands full. I worked at Central Point Software for awhile, writing Copy II 64/128 and I chatted quite a bit with the guy who was writing Copy II ST. He said it was a nightmare dealing with the WD1772 (which I later had to use with the Commodore 1571 and 1581 drives), and he was right! You were in a box without much control.
I did buy Turrican from the U.K. Hopefully it is the correct version, and I can test it with SuperCard Pro. Dungeon Master and Chaos Strikes Back copy just fine, and I am not altering any data from the data that is read - not even a normalization (write precomp) at this point, and the fuzzy bits are duplicated perfectly.
If strongbits are in fact a series of super fast transitions, then SuperCard Pro should be able to duplicate them. I believe I have the flux threshold set for 475ns. This means flux transitions can be this short before considered invalid. I realize that the idea behind the "no flux area" is that fact that the hardware's pulse detector can't do the normal return to zero, but there are drives (most PC drives) that don't have a problem showing the short pulses.
Posts: 27
Threads: 0
Joined: Dec 2013
It is a bit more complex than that and it will not work with a copier unless it is handled specifically. Let me try to explain what I believe is happening. (Note that I am not 100% sure because I do not have access to a good Scope/Logic analyzer …)
What is happening is that very fast transitions (unfortunately I do not know at what frequency) are recorded on the disk. This is possible because the write channel of the Floppy drive has no limitation (at least at that frequency). But when this transitions are read by the floppy drive read channel, by design (and even on modern drive) the signal is “integrated” (part for ACG, and I guess part for LF filtering) so what is happening when reading this burst of fast transitions is that you get *nothing* out of the read channel… So obviously if you try to blindly copy the transitions it won’t work. What has been done on some competitors products is that NFA are detected and when written back the area with no flux is converted to an area with very high flux rate. This protection is in fact used on many Atari games.
So this is already one case where blind copy won’t work.
In general blind bit copy works fine with a lot of games like DM CSB or Games using Rob Northern protection(s) on bit with variation (long/short sector or track). But many Atari games uses protections that I refer as Data over index and ID over index. As the name indicates an ID field or a DATA field is located just *over* the index and … On a normal track there is an area where the write head is turned on/off that I call the “track write splice” area. When you write a normal track the FDC wait for the index pulse and start to write the track until the next index pulse where it turn off the writing. Due to motor speed variation (and others…) the exact emplacement where the write gate is turned off is not precise at the bit flux level and this is the “track write splice” (there is also a “sector write splice” but this is another story). So if you have an ID segment or a DATA segment over the index and blindly copy from index to index you will always get glitches in the index area and the data won’t be correct. On professional duplication machine you could specify the location of the “beginning” of the track and writing would continue until the same point is reached again. Below you will see an extreme case where on some track only few sync bytes are located on the end of the track. In that case obviously blind copy won’t work either.
This is the only two obvious cases that come immediately to my mind and that would defeat a blind copy but they may be others. Usually pre-comp is not a problem as DPLL can handle this fairly well.
Now the good news is that once you have analyzed the protection it should be easy to reproduce it with your hardware.
Here is the Disk Layout for “Computer Hits Volume 2”. The ID field is in yellow and is followed by an intra-gap in light green followed by data in blue, followed by inter-gap in green. As you can see in that case the ID is just in front of the index and in some cases just above (track 70+) the index.
For example track 77 is 199822 µs long (not compensated) and if you look at the “read track buffer” (artificially prolonged) you can see the ID (line with byte 6272 starting at 199787 µs) that indeed span over the index.
Posts: 2,839
Threads: 148
Joined: Nov 2013
I understand what you are saying about not getting any change in the read line state for some period of time. In testing I have found that this is not the case on any drive I have tried. Maybe it was the case for the WD1772 controlled floppy drives. But like I said, I have deliberately tried ultra short flux transitions and there is no case that I can create where there is a NRZ state like you describe. The drives will always (at some point) change the state of the RDData line due to the AGC and adjacent track magnetic flux transitions being picked up. But, to be sure, I have Turrican coming for testing. Can you tell me what other Atari ST games use this same protection? I will also get those for testing.
I don't worry about data over an index. Using the Blind Mode OFF uses two full revolutions, so somewhere on the 2nd revolution is where the writing is turned off when the actual write splice is found. What games use this type of protection? I should get some for verification too!
You have to remember that I don't ever look at decoded data, only flux level data. With the Amiga copier (SuperCard Ami) I also never looked at decoded data, only the raw MFM data stream. So, I never deal with sectors and headers and such, just flux transitions and determining the physical location of where the head writing was turned off.
Posts: 27
Threads: 0
Joined: Dec 2013
12-20-2013, 10:02 AM
(This post was last modified: 12-20-2013, 10:08 AM by DrCoolZic.)
This is not related to WD1772 FDC! This is related to the FD Drive and not FD Controller
And no, no data comes from the read data line
This is the magic of this protection: as I said the read channel is in fact getting lots of transitions and therefore the ACG is kept at low amplification because it receives all these transitions but ... nothing is going out ...
If you look at the datasheets I have mentioned you will see that when transitions are getting closer the data bits detected are shifted. This protection "over-abuse" this behavior to the point that nothing comes out from the read channel (you can think of it as being filtered).
Quote:But like I said, I have deliberately tried ultra short flux transitions and there is no case that I can create where there is a NRZ state like you describe.
I do not understand what you mean?
As I said the write channel does not filter anything. So if your device is capable of "pulsing" the write data line at relatively high frequency you should be able to reproduce the protection. If you have a scope (unfortunately I dont) and Turrican you should be able to see what I mean.
Note that I got this information partially from Istvan but I have not been able to taste it because this would require a scope or Logic analyzer.
Quote:You have to remember that I don't ever look at decoded data, only flux level data. With the Amiga copier (SuperCard Ami) I also never looked at decoded data, only the raw MFM data stream. So, I never deal with sectors and headers and such, just flux transitions and determining the physical location of where the head writing was turned off.
This is quite interesting information. I now have to think about it to make sure this is always feasible with Atari protections. But great idea
Posts: 2,839
Threads: 148
Joined: Nov 2013
For Dungeon Master, Chaos Strikes Back, Chaos Strikes Back - Utility Disk, and a bunch of midi programs I have had seen no issue identifying the write splice. In fact, with ST and PC disks (those that use the typical ISO layout) it is much easier to identify the write splice because typically there are a lot of them throughout the disk.
I do have logic analyzers and scopes. So, I can (and have on a regular basis) dump the flux transitions and store the data. I have never seen a single case ever where there is no flux transition on the RDData line for any real extended period of time. The read line would have to be held high during this time.
My code currently can generate as low as a 875ns pulse width. I didn't think it needed to be shorter than that, but perhaps it does? I could make it as low as 100ns if needed. It would really help to have some samples of disks that used this type of protection. Is there a list of disks that use this protection as well as those that use the "data under index" protection?
|