The following warnings occurred:
Warning [2] Undefined property: MyLanguage::$archive_pages - Line: 2 - File: printthread.php(287) : eval()'d code PHP 8.1.27 (Linux)
File Line Function
/inc/class_error.php 153 errorHandler->error
/printthread.php(287) : eval()'d code 2 errorHandler->error_callback
/printthread.php 287 eval
/printthread.php 117 printthread_multipage



CBMSTUFF FORUM
Reference of first flux transition - Printable Version

+- CBMSTUFF FORUM (https://www.cbmstuff.com/forum)
+-- Forum: CBMSTUFF PRODUCTS (https://www.cbmstuff.com/forum/forumdisplay.php?fid=1)
+--- Forum: SuperCard Pro (https://www.cbmstuff.com/forum/forumdisplay.php?fid=3)
+---- Forum: Developer Area (https://www.cbmstuff.com/forum/forumdisplay.php?fid=13)
+---- Thread: Reference of first flux transition (/showthread.php?tid=868)

Pages: 1 2


Reference of first flux transition - AlWe - 12-21-2021

In my multi revolutions images something is off with the track length or my understanding of the format.

I'm just reading in the flux data, add them all up and then compare the result with the corresponding duration field from the TRK header.
Here what I e.g. get for one of my 1.44 MB IBM disks:




Quote:rev: 0, index2index=7987142, flux transitions=76847
rev: 1, index2index=7988418, flux transitions=76853
rev 0: acc ticks=7987094
rev 1: acc ticks=7988423



index2index and flux transitions are the duration and the bitcells from the TRK header, while acc ticks is the sum over all flux transitions in the revolutions.

For rev 0 there are 48 "free" ticks which should be the ticks from the last flux transition to the index signal,
But in rev 1 there are 5 ticks *more* than the revolution is long...

Noteworthy is also the discrepancy: while the index2index time varies 1276 ticks (31.9ms) the accumulated ticks have a discrepancy of 1329 ticks (33,225ms)
So the second revolution as roughly 2ms longer, and has 6 flux transitions more. Which seems to be too much to be explained away by some rounding errors.

As background why I'm looking at that:
There is at least one floppy formal - XDF - which is writing sectors over the index pulse and therefore I have to stitch revolutions together correctly, too.

My first assumption was, that the scp format is simply using the time from the first index pulse as reference for the first recorded flux transition.
The question therefore was, if I have to account for the time between the last flux in a track when I want to decode the first flux transition when reading over the index mark.
Now since the first flux transitions in my mult-rev images are always identical it looks like the reference may already be the last flux prior to the index signal. (Which would prevent us to figure out how far away the first flux transition is from the index pulse...)
That could explain why the sum of all flux transactions is longer than the track.

But I simply can't match the what I see into either assumption... So any tips what I'm doing wrong here?


RE: Reference of first flux transition - admin - 12-21-2021

The SuperCard Pro uses a hardware interrupt to start the capture and that comes from the index pulse.  The index to index time and total number of bitcells comes from the drive.  If you are seeing a shorter (or longer) duration from revolution to revolution then that would be the information that is provided by the drive itself.  It could be that the drive is really slow at spinning up or takes some time to transition from a step to a read.  Some really old 5.25" drives need a long delay after a step is issued before reading can occur. 

The number of bitcells can vary quite a bit depending on the number of weakbits (invalid flux) and how invalid flux is handled by your particular model disk drive. What disk drive are you suing?  Note that the number of bitcells can change from read to read of the same track because of invalid flux and how the disk drive handles it, so don't expect that value to be 100% consistent.  Drive speed variations will change invalid flux values.  Invalid flux can span anywhere from a single bitcell to multiple bitcells (thousands) depending on the flux.


RE: Reference of first flux transition - AlWe - 12-21-2021

(12-21-2021, 11:56 AM)admin Wrote: The SuperCard Pro uses a hardware interrupt to start the capture and that comes from the index pulse.  The index to index time and total number of bitcells comes from the drive. 

Let's stick to single-revolution images for now.
The main issue I have is, that when I add all flux times of one revolution together I can get a bigger value then what the header has for index2index. Not always but still quite often... At least 1/3rd of all tracks, based on my very small sample size of three images so far. (I also see that in an amiga scp image but there I only checked track 0.)

With your confirmation that the first flux bitcell is relative to the index that would then probably mean that it's not stopping exactly at the next index pulse but can add some flux transitions after the index at the end to a track. But that would then be up to maybe 3ms and definitely > 100 25ns ticks...

In that case I can simply drop the bitcells which would be located behind the 2nd index pulse and directly jump back to the first flux of the track and figure out what the real flux2flux interval would have been without the Index.

Maybe a simple ASCII of how that could look like:
Code:
INDEX
1st flux (ticks since INDEX)
2nd flux (ticks since last flux)
...
last real flux of track
INDEX                   
flux (probably ticks since last flux instead of INDEX)
last flux recorded for TRK

To get the time between the last real flux and the index I would have to add flux times together until it gets bigger than the index2index time, remove the last addition again and then subtract the accumulated time from the index2index time to get the time between the last real flux on the track and the 2nd INDEX.
Would be a bit unexpected but trivial to handle...

I'm using a Panasonic Matsushita JU-257A604P 1,44MB Floppy Disk Drive, powered via the supercard.
My test disks images for that are the disk 1 from Novel Dos 7 and a Copy of MS Dos 6.2. (Both images can be decoded without any error.)


RE: Reference of first flux transition - admin - 12-21-2021

(12-21-2021, 01:26 PM)AlWe Wrote: Let's stick to single-revolution images for now.
The main issue I have is, that when I add all flux times of one revolution together I can get a bigger value then what the header has for index2index. Not always but still quite often... At least 1/3rd of all tracks, based on my very small sample size of three images so far. (I also see that in an amiga scp image but there I only checked track 0.)

Sure - that's quite possible and perfectly normal.  If you use the editor/analyzer and click on the DISPLAY FLUX button you can see the information about the read including the total number of bitcells and total index time.


(12-21-2021, 01:26 PM)AlWe Wrote: With your confirmation that the first flux bitcell is relative to the index that would then probably mean that it's not stopping exactly at the next index pulse but can add some flux transitions after the index at the end to a track. But that would then be up to maybe 3ms and definitely > 100 25ns ticks...

No, it is stopping at exactly the index pulse.  In fact, reading starts and stops in the middle of a bitcell, which is why the first and last bitcell times are typically much shorter than "normal".. there is no extra bits.


(12-21-2021, 01:26 PM)AlWe Wrote: In that case I can simply drop the bitcells which would be located behind the 2nd index pulse and directly jump back to the first flux of the track and figure out what the real flux2flux interval would have been without the Index.

Maybe a simple ASCII of how that could look like:
Code:
INDEX
1st flux (ticks since INDEX)
2nd flux (ticks since last flux)
...
last real flux of track
INDEX                   
flux (probably ticks since last flux instead of INDEX)
last flux recorded for TRK

To get the time between the last real flux and the index I would have to add flux times together until it gets bigger than the index2index time, remove the last addition again and then subtract the accumulated time from the index2index time to get the time between the last real flux on the track and the 2nd INDEX.
Would be a bit unexpected but trivial to handle...

Of course none of this necessary. I don't think you understand how invalid flux works.  The bitcell times will be basically wrong for anything that is invalid.  The capture is right, but what the drive itself outputs depends on what type of drive it is and how THAT actually handles flux reversals that are not within it's normal clocking window. You can never just add up the bitcell times and have them the same for every revolution, or matching the index pulse. You can get close IF they are very few invalid bitcells. OLD disks that use sector writing (as opposed to entire track writing) will have a LOT of invalid flux. Every time you write to a track and turn off the writing, the head's magnetic energy collapses creating a "smear", which almost always results in invalid flux.


(12-21-2021, 01:26 PM)AlWe Wrote: I'm using a Panasonic Matsushita JU-257A604P 1,44MB Floppy Disk Drive, powered via the supercard.
My test disks images for that are the disk 1 from Novel Dos 7 and a Copy of MS Dos 6.2. (Both images can be decoded without any error.)

You might want to try a different disk drive, you are likely going to find different results.  Also, make sure that your USB port can handle the 300mA of current that the drive is going to need. However, what you are seeing is pretty typical for flux data. Some drives work better than others.

Are you working on a project using SuperCard Pro? If so, there is a LOT of code already written in C for handling every aspect of SuperCard Pro. You might be trying to re-invent the wheel. Smile


RE: Reference of first flux transition - AlWe - 12-22-2021

First a correction: I only have the issue with multi-rev images. And since multi-rev images can only be created in splice mode it could be that the splice mode is the issue here... Long story below, but I now suspect that I simply should ignore the index2index time when the image has been created in splice mode.

Quote:No, it is stopping at exactly the index pulse.  In fact, reading starts and stops in the middle of a bitcell, which is why the first and last bitcell times are typically much shorter than "normal".. there is no extra bits.
Are you saying that the last flux in a track is not the time up to the next flux change but to the index pule instead?

That's so unexpected I assume I'm simply overanalysing your statement and want to double check on that...
My best understanding still is, that I have to add all flux times of a track together and subtract the time from the index2index time to get the time between the last flux of the track to the index pulse. (At least when the image has been created in index mode.)

But for that the sum of all flux transitions (of one track and revolution) must be less or max equal to the index2index time of that track/revolution.
Do I somehow have to ignore invalid flux times for that? If so I'm really missing something fundamental here:

The drive is simple reporting flux changes and the supercard is making time stamps out of it. I see no way how a invalid long or short flux time reported by the drive could change the fact that the sum of all flux intervals for one track must be less or equal to the index2index time of the revolution. (I see how different revolution/reads can get different results but for each track/revolution. But not how adding up all flux intervals - with 0 interpreted as 0x10000 - can exceed the index2index time reported for the revolution.)

I re-imaged the disk, here a link to it: https://www.awhome.eu/index.php/s/bJ8NHznibT3DNfK
(I created three images in index mode, these are all ok. But from the two images in splice mode the one I linked to has the issue again.)
When I open the image with the SCP software the Flux view informs me, that the "Index" is 399538us and the "Cells"  are 399539.200us long for track 0 head 0.
Which make no sense: The Cell time is 1.200us longer than the two revolutions, so these are not all fitting within the disk revolutions.

Now there must be some rounding in the SCP software, which reduces the 1.2us "overrun" to just 0,075us. But I can't reason away those remaining 75ns.
Looking at the TRK header in hex:

Code:
000002b0  54 52 4b 00 6d ed 79 00  65 32 01 00 1c 00 00 00  |TRK.m.y.e2......|
000002c0  90 ee 79 00 70 32 01 00  e6 64 02 00 00 71 00 4b  |..y.p2...d...q.K|


Rev 1: 6d ed 79 00 -> 7990637 ticks (199765925ns)
Rev 2: 90 ee 79 00 -> 7990928 ticks (199773200ns)
Total: 15981565 ticks (399539125ns)
So the SCP flux viewer (399538us instead of 399539,125us) is not reporting 1,125us (aka 45 ticks) for the index2index time.
A bit unexpected but no big deal.

The problem here is, that even 399539.125us is still less than the 399539.200us the SCP flux viewer is reporting for the Cell times: We still have 75ns (3 ticks) more in cells then what can fit into the index2index time...
Thus my initial assumption, that the track end may not be exact. (Now it looks more like Splice mode is a bit lax when stitching revolutions together and simply should not be used when there is data written over the index.)

Decoding the same track with my code adds some additional data:
rev 0 has 199766.225us and rev 1 199772.975us worth of flux data.
Adding these together we also get 399539.200us for all flux transitions in total. Therefore the Cell time reported by the SCP Flux viewer must be correct and the discrepancy can't be explained by some rounding error here.


As for my project: I'm working on scp image support for qemu. I copied some core functions from fs-uae and also looked at the Disk Utilities (all code from Keir Fraser) for scp track loading and MFM decoding.
Still much to do but the read support for IBM MFM scp images is mostly done and working fine.
But while testing it I tripped over the fact that sometimes the sum of all flux transactions is not fitting into the corresponding index2index time, breaking the corner case support for reading a sector over the index.


RE: Reference of first flux transition - admin - 12-22-2021

(12-22-2021, 12:14 PM)AlWe Wrote: Are you saying that the last flux in a track is not the time up to the next flux change but to the index pule instead?

Yes, but only for the LAST revolution (when the capture ends).  So, the very first bitcelll will be a value from the index start position, which will be in the middle of that bitcell period.  On a multi-rev capture the data is uninterrupted for each of the revolutions, except the last revolution where the last bitcelll time will be cut short because the index pulse occurred.  SuperCard Pro is an interrupt driven hardware capture device.  The start and stop are triggered by the index pulse sensor.


(12-22-2021, 12:14 PM)AlWe Wrote: That's so unexpected I assume I'm simply overanalysing your statement and want to double check on that...
My best understanding still is, that I have to add all flux times of a track together and subtract the time from the index2index time to get the time between the last flux of the track to the index pulse. (At least when the image has been created in index mode.)

No.  You don't do that.  The index to index time is the ACTUAL time it takes for the rotation, down to the 25ns resolution.  It's 100% accurate, as are the bitcell times (reported by the drive - which could be wrong if the bitcell was invalid).  You don't add or subtract anything.  The data presented is an exact representation of what the disk drive sees.


(12-22-2021, 12:14 PM)AlWe Wrote: But for that the sum of all flux transitions (of one track and revolution) must be less or max equal to the index2index time of that track/revolution.
Do I somehow have to ignore invalid flux times for that? If so I'm really missing something fundamental here:

The drive is simple reporting flux changes and the supercard is making time stamps out of it. I see no way how a invalid long or short flux time reported by the drive could change the fact that the sum of all flux intervals for one track must be less or equal to the index2index time of the revolution. (I see how different revolution/reads can get different results but for each track/revolution. But not how adding up all flux intervals - with 0 interpreted as 0x10000 - can exceed the index2index time reported for the revolution.)

I re-imaged the disk, here a link to it: https://www.awhome.eu/index.php/s/bJ8NHznibT3DNfK
(I created three images in index mode, these are all ok. But from the two images in splice mode the one I linked to has the issue again.)
When I open the image with the SCP software the Flux view informs me, that the "Index" is 399538us and the "Cells"  are 399539.200us long for track 0 head 0.
Which make no sense: The Cell time is 1.200us longer than the two revolutions, so these are not all fitting within the disk revolutions.

It's possible because of drive speed variations and how invalid flux works.   When invalid flux occurs, the drive will spit out a bitcell duration that is somewhat based on the reversal time, and that is affected by the drive's speed.  I have seen many cases, especially with NFA (no flux area) protection schemes where the drive's PLL circuit takes serveral bitcells before it re-syncs, and I have also seen a lot of cases where flux reversals are actually joined together making a longer than normal bitcell time.  For the most part, it's pretty close in time (1.200us is obviously not much), but these are always 100% due to invalid flux.


(12-22-2021, 12:14 PM)AlWe Wrote: Now there must be some rounding in the SCP software, which reduces the 1.2us "overrun" to just 0,075us. But I can't reason away those remaining 75ns.

There is no rounding at all.  What you see is the information provided BY THE DISK DRIVE.  There is no interpretation here.  It's a very simple interrupt driven 25ns hardware capture.  The disk drive is solely responsible for the data you are seeing. In cases where you add up the bitcells and they exceed the index to index time, that is due to the drive speed variations. These drive speeds are all over the place during a revolution, and that really depends on the drive. Remember that these are using brushless multi-pole motors and every pole firing changes the drive speed slightly. In the grand scheme of things it doesn't matter, but when you are capturing at the 25ns level these are things you can see going on.


(12-22-2021, 12:14 PM)AlWe Wrote: Looking at the TRK header in hex:

Code:
000002b0  54 52 4b 00 6d ed 79 00  65 32 01 00 1c 00 00 00  |TRK.m.y.e2......|
000002c0  90 ee 79 00 70 32 01 00  e6 64 02 00 00 71 00 4b  |..y.p2...d...q.K|

Rev 1: 6d ed 79 00 -> 7990637 ticks (199765925ns)
Rev 2: 90 ee 79 00 -> 7990928 ticks (199773200ns)
Total: 15981565 ticks (399539125ns)
So the SCP flux viewer (399538us instead of 399539,125us) is not reporting 1,125us (aka 45 ticks) for the index2index time.
A bit unexpected but no big deal.

The problem here is, that even 399539.125us is still less than the 399539.200us the SCP flux viewer is reporting for the Cell times: We still have 75ns (3 ticks) more in cells than what can fit into the index2index time...

Yep, and this is not uncommon - just like it is not uncommon to see fewer bitcells compared to the total time, and if you do repeated reads you are going to find that the actual number of bitcells captured can actually change as well... all depending on how the drive handles the invalid flux is on the track.


(12-22-2021, 12:14 PM)AlWe Wrote: Thus my initial assumption, that the track end may not be exact. (Now it looks more like Splice mode is a bit lax when stitching revolutions together and simply should not be used when there is data written over the index.)

There is no "stitching" going on.  There is 512K of RAM on the SuperCard Pro board and the capture just fills the memory.  The index to index time is handle by a separate asynchronous interrupt.  That is why you get a table of index to index times separate from data stream.


(12-22-2021, 12:14 PM)AlWe Wrote: Decoding the same track with my code adds some additional data:
rev 0 has 199766.225us and rev 1 199772.975us worth of flux data.
Adding these together we also get 399539.200us for all flux transitions in total. Therefore the Cell time reported by the SCP Flux viewer must be correct and the discrepancy can't be explained by some rounding error here.

I don't think you realize what disk drives do with flux data.  The flux reversals are converted into a bitcell time.  The drive's clocking circuit is responsible for that.  There is a particular "window" that the drive can use and anything outside of that window is invalid and unpredictable.  This is why many PC disk drives can't handle writing (and sometimes even reading) Macintosh 400K/800K disk format.  Those use a variable speed drive so the reversals are not typically within the normal flux window.


(12-22-2021, 12:14 PM)AlWe Wrote: As for my project: I'm working on scp image support for qemu. I copied some core functions from fs-uae and also looked at the Disk Utilities (all code from Keir Fraser) for scp track loading and MFM decoding.

The source code for HxC is available, and that already has the support you are working on done.  Sam Owen has code available as well from SamDisk, there is also a8rawconv, and a slew of emulators with source code available.


(12-22-2021, 12:14 PM)AlWe Wrote: Still much to do but the read support for IBM MFM scp images is mostly done and working fine.
But while testing it I tripped over the fact that sometimes the sum of all flux transactions is not fitting into the corresponding index2index time, breaking the corner case support for reading a sector over the index.

This is all perfectly normal.   VALID data will always be correct.  It's the invalid data is what will throw off everything when you start analyzing to this degree of accuracy.


RE: Reference of first flux transition - admin - 12-22-2021

One other thing, if you have something like 100 invalid flux reversals in a row (not uncommon for protections and unformatted tracks) when you read that data it is going to change every single time. In fact, the first bitcell time could be 0x00A0 and the next could be 0x340, and the next time you read it they are reversed or some other value based on the bitcells around them. "Weakbit" copy protection uses this fact that the data changes on every single read. It is very common to see valid data placed among weakbit data and a check is made to make sure that the valid data is there and that also the invalid data changes on every single read. When you capture tracks like this you are going to see that the number of total number of bitcells on the track changes every read. Everything depends on the exact model drive you are using. Some disk drives simply return a fixed value when the clocking window is exceeded. Most drives will increase the AGC (automatic gain control) of the read amplifier to try to read the next bitcell period and continue to do that until valid data appears. This causes the over-saturation of the signal and the PLL circuit has to reset itself, which can mean losing several bitcells. NFA is particularly nasty because there is no flux transition that occurs at all for a period of time, up to a full revolution (166.6667 for a 360 RPM drive or 200ms for a 300 RPM drive). SuperCard Pro handles this as a special case using a 25ns synchronous clock.

I think you are believing that bitcell times are always perfect and consistent. This is not the case. The best case scenario is when a disk is produced on a commercial duplication machine with a master script so only a single write splice occurs. This is done by writing more than one full revolution so that when the write is turned off the smearing that occurs when the head's energy collapses only occurs one time during the revolution. When sectors are written individually, you have write splices (invalid flux) that occurs for every single header and data block, so you will have dozens of these on a track. The big dilemma in copying a disk is determining where to start/end the track. There are MANY cases where when you turn off the writing it just so happens to work out that the write splice (smear) ends up creating a bitcell value that is valid, so you can't see an obvious write splice. SuperCard Pro's SPLICE mode looks at the 2nd revolution (regardless of how many revolutions are captured), looking for an invalid flux time and uses that for stopping the writing when making a copy of the track. If there is no invalid value found, then SPLICE mode falls back to just writing the complete revolution, which is fine for anything that uses the index pulse to start/stop tracks.


RE: Reference of first flux transition - AlWe - 12-22-2021

First thanks for all the support!
You are right, I'm pretty new to flux images and all the nice details these have...
And I'm probably not using the correct terms in all cases, which may cause some confusion.
Here you seem to come back again and again to the amount of flux changes. But I do not understand how the flux ticks of one revolution added together can be bigger than the index2index time of the same revolution:

I thought I got at least the basics right: The disc drive reports any detected flux change via a line. It was news for me that the drive cares about the bitcells at all but I see how this AGG may need a reference to adjust itself within a window.
But the "interface" between the floppy drive and the supercard should be just one line which changes the state whenever the drive thinks it has found a flux transition. The supercard is ticking with 25ns and marks down the ticks between each state change of the floppy data line. And these state changes will later end up as the flux times of the track, with the amount of ticks counted between each state change as it's value.
Thus all timing information of a track revolution and also of the intervals between flux changes should come from the supercard pro, isn't it?

Assuming one revolution of the disk is measured by the supercard to be 7990637 ticks, the accumulated ticks for all logged flux transactions should not be able to exceed 7990637 ticks. While probably impossible I'm ok with zero flux changes during the full time. I'm can follow as long as all the flux data together equals any number between 0 and 7990637. But with one clock and exact start/stop there should be no way to get more than 7990637 ticks summing up all the individual flux times. (And for that we would have to detect a flux change at exactly 0 and 7990637 ticks.)
Of course things change when we have more than one clock: These will next to be sure out of sync and explain the derivations.

Bitcells are not on my radar for the topic here at all. But I understand that when we write e.g. a single sector on real HW it will never be in sync with the pre-existing data. When decoding we'll always have flux transitions outside of the "expected" bit cell. Which is the reason why we are writing the Syny Bytes (zeros) prior to the (I)DAMs, allowing the PLL to find the new bitcell window. In reality the bit cell window and also the clock is shifting around all the time even when we are in sync.

Nevertheless the explanation of the Index mode may be what I was looking for, assuming it also can stop reading after the index mark when creating a image.
After all all the images I recorded with Index seems to be correct, the sum of all the flux ticks is less that the index2index time.
For the three images of the same disk I got those values:

Index2Index  sum of all flux times
199783600 199782750 -> 850 free ticks
199773900 199772425 -> 1475 free ticks
199775275 199774425 -> 850 free ticks

Which seems to underline that the Index2index time can't be aligned to individual flux ticks: 21-37us is a serous offset, especially when the last flux id for the index again. 
Would it be safe to say that the Index2index time can only be used to determine the rotation speed and and can't be used for anything other?

I also should point out that according to this discussion at least fs-uae is decoding the first flux bit wrong : The code is not handling the last flux transition as mark for the index pulse and this should not be able to read an XDF image from my point of view.


RE: Reference of first flux transition - admin - 12-23-2021

(12-22-2021, 03:50 PM)AlWe Wrote: First thanks for all the support!
You are right, I'm pretty new to flux images and all the nice details these have...
And I'm probably not using the correct terms in all cases, which may cause some confusion.
Here you seem to come back again and again to the amount of flux changes. But I do not understand how the flux ticks of one revolution added together can be bigger than the index2index time of the same revolution:

I thought I got at least the basics right: The disc drive reports any detected flux change via a line. It was news for me that the drive cares about the bitcells at all but I see how this AGC may need a reference to adjust itself within a window.
But the "interface" between the floppy drive and the supercard should be just one line which changes the state whenever the drive thinks it has found a flux transition. The supercard is ticking with 25ns and marks down the ticks between each state change of the floppy data line. And these state changes will later end up as the flux times of the track, with the amount of ticks counted between each state change as it's value.
Thus all timing information of a track revolution and also of the intervals between flux changes should come from the supercard pro, isn't it?

Yes, that is correct and that information is what you are seeing.  It would be great if we could just add up all of the bitcell times and that would match the index to index times, but that's not how it works in the real world. However, it doesn't even matter because the index to index time is never needed unless you are trying to re-image the disk at a different RPM. That is the ONLY time you are going to want to know the index to index time so you can adjust ALL of the bitcells (up or down) in "speed" so it fits within the track time you are trying to obtain.  The drive itself is what outputs a pulse of a certain duration that occurs when a flux reversal occurs.  The issue is that if a flux reversal does not occur within a preset window, then the results are unpredictable.  The drive can output multiple (extra) pulses or fewer pulses, depending on what the drive circuitry is designed to do when the window is exceeded.

Depending on the drive (which is the only real factor here), it will handle the flux differently.  Also, keep in mind that the start/stop is being done using the index pulse and the very first bitcell of the 1st revolution and the very last bitcell of the last revolution will be affected by the index pulse because the capture start and stop occur with the index pulse.  So, because of this alone there could be an increase or decrease in the number of bitcells due to drive speed variances and where the index pulse triggers. For disks that were creating using the index pulse to start/stop tracks, it doesn't matter if the first few hundred bytes (not even bitcells) are wrong because they are contained in the track gap. It's only disks that were not created using the index pulse to start/stop the writing is where you have to get creative and use 2+ revolutions to be able to extract one full interrupted revolution.


(12-22-2021, 03:50 PM)AlWe Wrote: Assuming one revolution of the disk is measured by the supercard to be 7990637 ticks, the accumulated ticks for all logged flux transactions should not be able to exceed 7990637 ticks.

Sure it can!  The drive will output whatever it thinks is valid flux reversals, when in fact it could be invalid flux reversals, and the drive speed is not consistent. You don't see a huge increase, but up to several microseconds is not uncommon by any means.


(12-22-2021, 03:50 PM)AlWe Wrote: While probably impossible I'm ok with zero flux changes during the full time. I'm can follow as long as all the flux data together equals any number between 0 and 7990637. But with one clock and exact start/stop there should be no way to get more than 7990637 ticks summing up all the individual flux times. (And for that we would have to detect a flux change at exactly 0 and 7990637 ticks.) Of course things change when we have more than one clock: These will next to be sure out of sync and explain the derivations.

You seem to be hung up on the fact that you can't simply add the bitcell times together and magically come out to be exactly the index to index time.  You need to get out of that way of thinking, because you have no control of what the disk drive is going to do.  Adding the bitcell times together is *rarely* ever going to be equal, and you can certainly have a value that is longer or shorter than the index to index times.   There is never any reason to even know this unless you are writing some utility like my editor/analyzer. You don't need to know anything other than there is a block of flux data and what your upper level format is (MFM, and then decode that to usable hex data).


(12-22-2021, 03:50 PM)AlWe Wrote: Bitcells are not on my radar for the topic here at all. But I understand that when we write e.g. a single sector on real HW it will never be in sync with the pre-existing data. When decoding we'll always have flux transitions outside of the "expected" bit cell. Which is the reason why we are writing the Syny Bytes (zeros) prior to the (I)DAMs, allowing the PLL to find the new bitcell window. In reality the bit cell window and also the clock is shifting around all the time even when we are in sync.

Oh... you are working with the really upper format.  You are better off looking at things from a MFM perspective because there are caveats with the FDC interpretation of the MFM data that is going to drive you nuts. Smile


(12-22-2021, 03:50 PM)AlWe Wrote: Nevertheless the explanation of the Index mode may be what I was looking for, assuming it also can stop reading after the index mark when creating a image.
After all all the images I recorded with Index seems to be correct, the sum of all the flux ticks is less that the index2index time.
For the three images of the same disk I got those values:

Index2Index  sum of all flux times
199783600 199782750 -> 850 free ticks
199773900 199772425 -> 1475 free ticks
199775275 199774425 -> 850 free ticks

Which seems to underline that the Index2index time can't be aligned to individual flux ticks: 21-37us is a serous offset, especially when the last flux id for the index again. 

21us-37us variation is just a couple of bytes of variances at the MFM level.  That's quite acceptable.  37us out of 200,000us (a 300 RPM rotation) is 0.0185% deviation.  Consider that a track gap for the type of disk you are working with is typically 200 MFM bytes (800us-1000us long), having a 37us variation over a 200ms period is not going to be a problem for the PLL.


(12-22-2021, 03:50 PM)AlWe Wrote: Would it be safe to say that the Index2index time can only be used to determine the rotation speed and and can't be used for anything other?

That's all it is for!  I actually don't use that information myself for reads. I am not aware of ANY program supporting .scp images that does.  For writing or converting you need to know that. What did you expect it to be used for?  You can use it to account for drive speed variations between image files, but seriously, if you are just decoding disk images for an emulation you can be so far off and everything will still work. You can delay the reading of a track from the index pulse by several hundred microseconds and still be able to read/decode the entire track correctly. Keep in mind that the decoding window for a standard FDC is +/- 8% of the bitcell times. So, if you have a typical ISO format (IBM PC, Atari ST, TRS-80, etc. etc.) that uses 4us/6us/8us your bitcells can be off by +/-320ns on the low end (4us) and +/-640ns on the high end (8us). Nothing is critical with disk data until you get into the 2.88MB format and some DAT tape formats where sub-100ns accuracy can be required.


(12-22-2021, 03:50 PM)AlWe Wrote: I also should point out that according to this discussion at least fs-uae is decoding the first flux bit wrong : The code is not handling the last flux transition as mark for the index pulse and this should not be able to read an XDF image from my point of view.

Well, it's worked fine for years.  Smile XDF tracks start and stop on the index pulse, and there is a sizeable gap so there should be zero issues.


RE: Reference of first flux transition - AlWe - 12-23-2021

(12-23-2021, 12:20 AM)admin Wrote:
(12-22-2021, 03:50 PM)AlWe Wrote: Would it be safe to say that the Index2index time can only be used to determine the rotation speed and and can't be used for anything other?

That's all it is for!  I actually don't use that information myself for reads.  I am not aware of ANY program supporting .scp images that does.  For writing or converting you need to know that.  What did you expect it to be used for?  You can use it to account for drive speed variations between image files, but seriously, if you are just decoding disk images for an emulation you can be so far off and everything will still work.  You can delay the reading of a track from the index pulse by several hundred microseconds and still be able to read/decode the entire track correctly.  Keep in mind that the decoding window for a standard FDC is +/- 8% of the bitcell times.  So, if you have a typical ISO format (IBM PC, Atari ST, TRS-80, etc. etc.) that uses 4us/6us/8us your bitcells can be off by +/-320ns on the low end (4us) and +/-640ns on the high end (8us).  Nothing is critical with disk data until you get into the 2.88MB format and some DAT tape formats where sub-100ns accuracy can be required.

My assumption was, that the first flux is relative to the index and the last flux would be simply the last flux transition reported by the drive.

And by subtracting all flux time values from the index2index time I can determine the flux-free time between the last flux and the index again. (And adding that time to the value of the first flux transition again would then lead me to the flux2flux gab between the last and the first flux of one revolution.)

For multi-revolution images this would have been allowed to pinpoint the exact time we got the index signal. With the real format we only know that the index pulse must have been between two recorded flux times but can only pin down the exact time for the last revolution. (Since we still get the time between the last flux of rev 1 and the first flux of rev 2 there is no real downside I see on that. Just not what I assumed...)

(12-23-2021, 12:20 AM)admin Wrote:
(12-22-2021, 03:50 PM)AlWe Wrote: I also should point out that according to this discussion at least fs-uae is decoding the first flux bit wrong : The code is not handling the last flux transition as mark for the index pulse and this should not be able to read an XDF image from my point of view.

Well, it's worked fine for years.  Smile  XDF tracks start and stop on the index pulse, and there is a sizeable gap so there should be zero issues.

I'm still pretty new to flux images... but the sample XDF disk I found in the forum here https://www.cbmstuff.com/forum/showthread.php?tid=603&pid=3908#pid3908 seems to have data written over the index, see the attached picture. (It looks at least possible that the crc errors of the sectors in this image are caused by the unusual fact that we have date written over the index pulse and either the supercard or HxC is not handling that correctly. Once I get simpler images working I'll plan to look into that more.)