xx3dsfml Multi-platform N3DSXL Capture Software Audio Update

[static_zero]

Hi all - It's been a while since I've last been here. After hitting a wall with my capture software previously, I ended up becoming very busy with life and such, but things have finally started to slow down for me now. I've since been able to revisit my code and have made a major update in finally adding native audio support to it. To those of you who might have been interested in this, sorry for the long wait! The updated code with instructions for use with UNIX based systems can be found here.

Just as before, I've tried to keep the code as lean as I reasonably could, and performance seems to be quite good. That being said though, I did encounter a noticeable issue with the audio implementation in the form of playback latency. Since I was already using SFML for the video, I chose to use it for the audio as well, and I'm not entirely sure, but it seems to me at least like it may be what's causing the latency issue. To put it short, even when interfacing with the SFML SoundStream with the least amount of overhead and in the most direct way possible (even going so far as to temporarily remove any considerations for variable length sample size per read), I still found that the audio was slowly falling out of sync with the video no matter what I did. Mind you, the video plays back quite fast, almost having no perceivable latency to me at least, which is what leads me to believe it might be the SFML SoundStream implementation itself that's causing the majority of the audio latency.

My solution to get around this without interfering with the consistency of the audio too much was to just introduce a "tolerance" level for how much the audio could get out of sync and drop a read's worth of samples when it's exceeded. This does mean that there is still some amount of audio latency and that there's also some audio loss now, but I seem to have manged to balance it to where the audio isn't too out of sync but also the frequency of dropped samples isn't too often either. Based on what I've observed so far, the audio performance is still very reasonable and the software is more than usable like this. I really would like to have complete and consistent audio though, so I will be looking into ways that I might be able to improve this (or hopefully just fix it outright) and will continue to post about my progress.

Anyway, thanks again to anyone whose taken an interest in this. I may have made this software out of my own need, but I'm glad if even just one other person's getting something out of it!

[sportzfrk]

Hey man I just want to say great job on this!!!!

If you look at my fork (jsillick) I made a switch to using CMake to build as that build system is much more robust. It can basically take SFML version you want and build it from scratch with the main cpp file. CMake is free and available for all platforms.

I got your software to build on a Raspberry Pi 5 and it works great. In the future I’m going to try to make this more like Optimize’s “TV player” thing which was custom software on a Raspberry Pi. Would hopefully make it as a total image you just flash to a SD card and go. Idk yet, still a lot of thinking and I don’t have a ton of time to devote to this.

These are the packages it recommends to install to build SFML for Debian based OSes if you do it the CMake way.

sudo apt update
sudo apt install \
libxrandr-dev \
libxcursor-dev \
libudev-dev \
libopenal-dev \
libflac-dev \
libvorbis-dev \
libgl1-mesa-dev \
libegl1-mesa-dev \
libdrm-dev \
libgbm-dev

So those plus “CMake”. I believe most, if not all flavors of Linux have these packages on their package manager, maybe not the exact names though. I also believe the Mac Homebrew software has these as well, but cannot confirm.

[Chroma]

Thanks for your hard work! I'd need to spend a good off day familiarizing myself with building this since I'm more used to running pre-built executables, but "multi-platform" is getting me super excited that I might finally get something usable running on my Mac.

[static_zero]

Hey man I just want to say great job on this!!!!

If you look at my fork (jsillick) I made a switch to using CMake to build as that build system is much more robust. It can basically take SFML version you want and build it from scratch with the main cpp file. CMake is free and available for all platforms.

I got your software to build on a Raspberry Pi 5 and it works great. In the future I’m going to try to make this more like Optimize’s “TV player” thing which was custom software on a Raspberry Pi. Would hopefully make it as a total image you just flash to a SD card and go. Idk yet, still a lot of thinking and I don’t have a ton of time to devote to this.

These are the packages it recommends to install to build SFML for Debian based OSes if you do it the CMake way.

sudo apt update
sudo apt install \
libxrandr-dev \
libxcursor-dev \
libudev-dev \
libopenal-dev \
libflac-dev \
libvorbis-dev \
libgl1-mesa-dev \
libegl1-mesa-dev \
libdrm-dev \
libgbm-dev

So those plus “CMake”. I believe most, if not all flavors of Linux have these packages on their package manager, maybe not the exact names though. I also believe the Mac Homebrew software has these as well, but cannot confirm.

Hey, thanks bud!

For something that's <300 lines of code, it took a bit more work than I would have expected, but I'm glad to know that at least someone else is getting to use it.

I haven't gotten a chance to check GitHub in a while, but I am familiar with CMake. I've used it in the past on Windows for some of my own projects and to build other people's projects as well. It's pretty handy to be able to build SFML from source directly. I actually had the issue that SFML was out of date in my distro's packages repository when I came back to developing this, and the setProcessingInterval call for the audio stream wasn't available to me. I had to update the package myself just to get it (and I didn't even get to contribute it in the end ).

It's also cool to know that it's running on a Pi, and it sounds like a really neat project too! I'd love to see it someday if you ever get around to fleshing it out (I know how it can be with these projects sometimes ), but if you ever need a hand or some advice, I'd be happy to help.

[static_zero]

Thanks for your hard work! I'd need to spend a good off day familiarizing myself with building this since I'm more used to running pre-built executables, but "multi-platform" is getting me super excited that I might finally get something usable running on my Mac.

It's my pleasure! It would be nice to be able to provide binaries for it, but it's not very practical for small projects like this unfortunately. There was someone else who posted on my GitHub repository about having an issue building the software on Mac, but I hadn't heard back from them. If you have any issues yourself, I'd be happy to try to troubleshoot it for you.

Ideally, I'd like to test this out on a Mac myself and provide instructions for it, but I don't have a working one readily available to me at the moment (which may change in the future). Heck, I'd even like to see this working on Windows (even though there's really no need for it ). Being that FTDI's driver and SFML are available on all 3 platforms, there's no reason why it shouldn't be able to run everywhere.

[static_zero]

I also just wanted to add a couple of things here.

First off, I think I might have jumped to conclusions about SFML being responsible for the audio latency. While it may still be the case, I've also come to realize that I experience audio latency in a similar fashion when capturing games through my internal capture card in OBS. This is something that's platform independent actually because I remember having this same exact issue back when I was still using Windows with this same exact hardware. I suspected that it was just a simple mismatch of sample rates, but it doesn't seem to matter what I have it set to.

Just like with my 3DS capture software, the audio starts in sync and slowly falls out of sync over time until I refresh the device. Whatever inconsistency is causing the issue with my capture card and OBS may actually be a hint as to what's happening with my software as well. I haven't actually made any progress in solving this yet, but hopefully it will lead to more improvements in my software if and when I get to the bottom of it.

Anyway, I'm also planning on streaming for the very first time tomorrow (smooth, shameless plug) and finally getting to use the software I made for something! Really I'd like to just test it out for an extended amount of time across different games just to see how it performs. There's a lot of stuff that I'll probably be testing out, being my first stream and all, but if anyone hops in from here and wants to see 3DS/DS stuff, I'd be happy to show it off!

Link to my channel: https://www.twitch.tv/static_zero (I'll post again on here when I go live if I can remember and hopefully it won't be a complete train wreck )

[koba]

Thanks for your hard work! I'd need to spend a good off day familiarizing myself with building this since I'm more used to running pre-built executables, but "multi-platform" is getting me super excited that I might finally get something usable running on my Mac.

Ideally, I'd like to test this out on a Mac myself and provide instructions for it, but I don't have a working one readily available to me at the moment (which may change in the future).

I've actually been struggling to figure out how to get this working on Mac tonight as it turns out LOL, I added onto an issue on the github repo so hopefully there might be some way to get it working, I do love the idea of this being able to run multi-platform- finally we won't be limited to only being able to reliably stream our capture cards on Windows lol

[static_zero]

Thanks for your hard work! I'd need to spend a good off day familiarizing myself with building this since I'm more used to running pre-built executables, but "multi-platform" is getting me super excited that I might finally get something usable running on my Mac.

Ideally, I'd like to test this out on a Mac myself and provide instructions for it, but I don't have a working one readily available to me at the moment (which may change in the future).

I've actually been struggling to figure out how to get this working on Mac tonight as it turns out LOL, I added onto an issue on the github repo so hopefully there might be some way to get it working, I do love the idea of this being able to run multi-platform- finally we won't be limited to only being able to reliably stream our capture cards on Windows lol

Haha, nice timing! I just replied to the issue on my GitHub repository. Let me know what you've done and how it goes, and hopefully we'll manage to get it working. It would be nice to get a confirmation that this is working on Mac and possibly be able to provide clearer instructions for other Mac users to get it working on their systems as well. fingers crossed

[static_zero]

I also just wanted to add a couple of things here.

First off, I think I might have jumped to conclusions about SFML being responsible for the audio latency. While it may still be the case, I've also come to realize that I experience audio latency in a similar fashion when capturing games through my internal capture card in OBS. This is something that's platform independent actually because I remember having this same exact issue back when I was still using Windows with this same exact hardware. I suspected that it was just a simple mismatch of sample rates, but it doesn't seem to matter what I have it set to.

Just like with my 3DS capture software, the audio starts in sync and slowly falls out of sync over time until I refresh the device. Whatever inconsistency is causing the issue with my capture card and OBS may actually be a hint as to what's happening with my software as well. I haven't actually made any progress in solving this yet, but hopefully it will lead to more improvements in my software if and when I get to the bottom of it.

Anyway, I'm also planning on streaming for the very first time tomorrow (smooth, shameless plug) and finally getting to use the software I made for something! Really I'd like to just test it out for an extended amount of time across different games just to see how it performs. There's a lot of stuff that I'll probably be testing out, being my first stream and all, but if anyone hops in from here and wants to see 3DS/DS stuff, I'd be happy to show it off!

Link to my channel: https://www.twitch.tv/static_zero (I'll post again on here when I go live if I can remember and hopefully it won't be a complete train wreck )

After struggling a bit with setting up some last minute homebrew to get Majora's Mask 3D (and other titles) working on my JP 3DS, I've finally went live. Feel free to stop in and hang out or just come and say hi!

[static_zero]

I also just wanted to add a couple of things here.

First off, I think I might have jumped to conclusions about SFML being responsible for the audio latency. While it may still be the case, I've also come to realize that I experience audio latency in a similar fashion when capturing games through my internal capture card in OBS. This is something that's platform independent actually because I remember having this same exact issue back when I was still using Windows with this same exact hardware. I suspected that it was just a simple mismatch of sample rates, but it doesn't seem to matter what I have it set to.

Just like with my 3DS capture software, the audio starts in sync and slowly falls out of sync over time until I refresh the device. Whatever inconsistency is causing the issue with my capture card and OBS may actually be a hint as to what's happening with my software as well. I haven't actually made any progress in solving this yet, but hopefully it will lead to more improvements in my software if and when I get to the bottom of it.

Anyway, I'm also planning on streaming for the very first time tomorrow (smooth, shameless plug) and finally getting to use the software I made for something! Really I'd like to just test it out for an extended amount of time across different games just to see how it performs. There's a lot of stuff that I'll probably be testing out, being my first stream and all, but if anyone hops in from here and wants to see 3DS/DS stuff, I'd be happy to show it off!

Link to my channel: https://www.twitch.tv/static_zero (I'll post again on here when I go live if I can remember and hopefully it won't be a complete train wreck )

After struggling a bit with setting up some last minute homebrew to get Majora's Mask 3D (and other titles) working on my JP 3DS, I've finally went live. Feel free to stop in and hang out or just come and say hi!

I'm happy to say that the stream went well yesterday. I was live for about 6 hours playing Majora's Mask 3D and Kirby Super Star Ultra, and there were no issues with my software. I even played out of my OBS preview mostly, so it's definitely good when it comes to latency. All in all, I'm happy with the results, but I'll still be trying to get to the bottom of the audio issue and possibly working on other enhancements/features too.

I'm also live again right now just for fun, so I figured I'd drop that in here as well.