xx3dsfml Multi-platform N3DSXL Capture Software (Soft) Release

[static_zero]

Hey Everyone - I'm (somewhat) happy to announce the release of my multi-platform capture software for the recently released New 3DSXL capture card which can be found here. The reason why I'm only (somewhat) happy is that while this software is perfectly usable, it's currently in an incomplete state, and I've run out of viable options for completing it at this point. There's really 3 issues with it as of now:

1. It runs inconsistently between 30 and 60 fps. This is due to the fact that it's taking between 1 to 2 reads to get a full frame of data over usb.
2. There are occasional misaligned/miscolored frames. These are very infrequent but do still happen and really shouldn't. These seem to only happen when it takes 3 (or presumably more) reads.
3. There's no audio support.

I believe that all of these issues could likely be solved if I had more information about how the capture card is controlled. I listened in on the data packets being transferred while using the official software on Windows and noticed some things about it:

1. When the software is actively capturing, it's consistently receiving 520615 bytes every read. In my software, using the default pipeline state, I can only read at most 518400 bytes (a full video frame) and it takes 1 to 2 but sometimes 3 reads to attain. Based on my observations, the additional 2215 bytes that I'm unable to get likely contains the audio data. Figuring out how to get the software/capture card in a state where I can get the full 520615 bytes every transfer is the key to solving the above 3 issues.

2. Whenever the software starts actively capturing, data is first written over usb to the capture card. This is the part where I'm pretty much at a loss because even though I can see what bytes are being transferred and when, I can't actually determine what they mean. The most noteworthy of these is that there are several 20 byte transfers to control out endpoint 0x01 that seem to refer to data in endpoint 0x82 which is where the data is captured over. I imagine what this is doing is giving the capture card instructions on how to start transmitting data, but I'm not sure how much of this is being done in or by the FTDI driver and how. There's also data being sent to data in endpoint 0x02 which I believe also plays a role in this somehow since I think this endpoint could have been omitted in the configuration if it wasn't actually needed for something (which means it's probably needed for something :p).

I haven't actually tried to submit any data over either of the out endpoints, but I might start playing around with this just to see if I can get any different behavior out of the data in endpoint. Best case scenario, I somehow manage to come up with something on my own, but in reality, I'll likely need some guidance to figure this one out (this is really the only likely way that an ideal implementation will ever be achieved to be honest).

For the meantime, I'm at least (somewhat) happy to put something working out there for anyone who might benefit from it. :]

[loopy]

The extra i/o at the start does some initialization and firmware version checking. Not strictly necessary, but you should at least write {0x40, 0x00, 0x00, 0x00} to EP2, which will reload the FPGA with the most recent configuration.

For the rest - you need to pull out all the performance tricks you can. That means:

• Change your RGB_FRAME_SIZE to include the extra audio data. You should be able to retrieve everything with a single read. (If you don't, just throw it out. It's going to be a garbled frame anyway)
• Do overlapped reads properly. That means grab multiple buffers with FT_InitializeOverlapped. Then start reads on all of them (FT_ReadPipeEx). Then keep the stream going using FT_GetOverlappedResult + reissue FT_ReadPipeEx on it. I think FTDI provides sample code for doing all of that.
• Probably best to do all of that in a separate thread.

[static_zero]

Thanks for the input Loopy. I've modified my code and am now getting a full 60 fps and no error frames. However, I'm still not able to read more than 518400 bytes in a single read, and I think I might know why. The issue might have to do with how the read functions are implemented in the D3XX driver outside of Windows. On Windows, you have access to an FT_ReadPipeEx function that takes a PipeID as the second input and an Overlapped as the last input (this can be seen in the ftd3xx.h file within an if defined(_WIN32) || defined(_WIN64) condition). Outside of Windows, there are two other functions that replace it: an FT_ReadPipeEx that takes a Timeout as its last input and an FT_ReadPipeAsync which does actually take an Overlapped as its last input.

However, the thing with these two functions is that they don't take a PipeID as the second input but instead take a FIFO Channel ID. In my code, I pass 0 as the channel to the FT_ReadPipeAsync function as this is the channel that contains the 0x82 pipe, but something about how this function is implemented (perhaps due to the fact that it's working on the whole channel and not just the one endpoint) seems to be preventing any reads beyond the frame data. This almost seems like too much of a coincidence that it's limited to exactly an even 518400 (that's why I was convinced up until yesterday that there had to be something more to this), but I don't really know what else could be causing it. It doesn't seem to matter how big I set the buffer size and buffer length to, it always ends up with this result. I'll go through everything again just to make sure that I wasn't making a mistake anywhere and unintentionally limiting it to that amount (I reduced the value in my uploaded code back to 518400 just for the sake of the release), but in the meantime, the uploaded code is at least running stable and smoothly now for anyone who might be interested.

[static_zero]

Hey, just wanted to give an update on this, I decided to test my suspicion out by writing a small program on Windows using the FT_ReadPipeEx call just to see how much data I could get out of it, and it turns out that I'm still only getting 518400 bytes each read. That being the case, the differences in the D3XX driver between platforms isn't what's causing my issue then and there must be something else that I'm missing.

I'm sure this is the last thing that you care about right now with the next batch of boards on the horizon, but I'm wondering if you have any other suggestions about what I could potentially do to get beyond the frame data each read. I've attached my code if you wouldn't mind taking a look at it or possibly trying to run it yourself (there's no rendering in it, it just prints the number of bytes that it's reading). One of the two in_buf variables needs to be uncommented before compiling, and you may also need to change the ftd3xx.h include since I linked everything dynamically on my end.

Also just to mention, it's not anything to do with my computer/ports/cable/3ds/etc. since your software still works without issue for me.

[loopy]

Try experimenting with BUF_SIZE. (As in, make it much larger than it needs to be).

[static_zero]

Well that did the trick, changing it to (RGB_FRAME_SIZE * 2) is giving me 520588 bytes every read (occasionally 520592). I just switched over to Linux and tested it out in my actual code and got the same result which is fantastic news, so now I'll see what I can manage to work out over the weekend with this. Thanks again for the help, it's such a simple thing that you'd think I would have just tried it on my own :p but I'm really glad to be back on the right track with this so I really do appreciate it!

[loopy]

For audio stuff, this might be useful. https://github.com/Gotos/CuteCapture/issues/2

I wouldn't spend too much time perfecting it though. I plan on changing it sometime in the near future (doing sample rate conversion onboard, instead of outputting the 3DS's janky nonstandard audio raw)

[static_zero]

Thanks, I'll take a look through to see what I can work out. It may take a bit to get to though because I just encountered another issue actually. Here's what happened:
1. Requested large read on Windows using FT_ReadPipeEx and got the full data.
2. Requested large read on Linux using FT_ReadPipeAsync and got the full data.
3. Unplugged and re-plugged the 3ds back into the computer.
4. Requested large read again on Linux and only got 518400 bytes.

What's happened I think is that issuing the large read on Windows got it in a state where it was outputting the full amount of data and since it never lost power in between the switch to Linux, it was still in that state and was still giving the full amount. Once I unplugged it though, it lost that state and is now unable to get the full amount again no matter how big of a buffer I use.

I noticed that while it was in the increased data throughput state, I was able to lower the buffer all the way down to exactly 520592 and still got the full data out of it. It seems like it just needs a single large read request to get it in that state, but using the standard FT_ReadPipe call to endpoint 0x82 doesn't seem to do it unfortunately. Would there possibly be another way to signal the capture card to start outputting the larger amount of data without using a read request? If not, I might need to work something out where I send requests over usb without using the D3XX api just to get it in that control state at the start. Probably won't be easy, but it might be the only option unfortunately..

[loopy]

Are you just trying to do a single read? It's normal for the first couple of frames to come out bad, once you've got a stream going it should even out.

[static_zero]

No, I tried to do at least 8 reads in a row before passing it over to the actual render pipeline, but it turns out the whole thing was just a red herring all along anyway because my Windows code doesn't actually get more than the 518400 bytes of frame data either, even with the large buffer. What *actually* happened was that at some point I booted up your software without disconnecting the 3ds from the computer after, and that was what actually got it running in the right state and made both my Windows code and Linux code work (and then not after I disconnected it). So I'm pretty much back to square one with this, meaning that there's something I'm not doing right regardless of what platform I'm on that's keeping me from getting the extra data, but I still have no inkling of what it could be..