Upgrading my GPU to a RTX5060

Upgrading my GPU to an RTX 5060 TI 16Gb

Previously I wrote about converting pdfs to epub using my aging GTX 970 with just 4Gb of Vram. The software I use is Marker which requires so much Vram that I had to build a custom framework to first split the PDF in seperate files, then convert the pdf files into .md files and finally the framework stitches it back together. This worked, but it is cumbersome. Furthermore as the software changed, I the number of pages that could be contained in pdf shrank to one and even with just one page some pages were too much for the aging GPU.

So I finally took the step to get a new GPU with lots more Vram, especially since prices for GPUs with lots of Vram are expected to rise. I got a GPU with an Nvidia RTX 5060 TI with 16 GB. So here is my experience

Massive improvement

I was expecting to be able to process more pages per batch and a lot faster. So when I got the new GPU working (see later for the trouble I had getting the card to work) the improvement was a lot more then expected. I started with a test file of 10 pages. As the 4Gb card could only handle one page, I expected this to be too much. But it handled it just fine. I ran nvtop during the conversion and something odd happened: the Vram memory usage plateaued at 50%. The compute part of the GPU did not even quite reach 100%. It got stuck at roughly 95%. A quick check with htop confirmed that of all the cores in the CPU, just one was really working and at 100%.

So eventually I thought, let’s see if it can handle the entire book in one go. And it did!. So the framework is no longer required. With a properly installed Pytorch and drivers it now no longer needs the framework. Obviously it also runs quite a bit faster :-).

So obviously no longer having so little Vram helps. But I do suspect other factors are in play as well. Since previously running on CPU with plenty of Ram (48Gb) would also fail. Other factors:

• With the newer GPU I could install Pytorch with Cuda version 13. The older, GTX, card only worked with older Cuda drivers
• Pytorch and Marker were also upgraded,

So all in all a big improvement. Unless a PDF is just a one off for you, it is well worth the investment.

Choosing a card, Nvidia/AMD/Intel

I run Linux on my PCs and have been doing this since ca. 2002. So while I am no Linux Guru, I’m not a newbie either. Now I am aware that Nvidia is known to be more difficult to get running properly on Linux then AMD or Intel. AMD supplies its drivers Open Source which means it is easier for Linux distributions to integrated properly. Nvidia drivers for Linux, as supplied by Nvidia, require extra steps to get installed, depending on your distribution. To be fair, I never experienced much of an hassle to get the Nvidia drivers to work in the past. The software I use is based on Pytorch. According to the documentation: Both AMD and Nvidia are well supported. Intel GPUs also apparently support Pytorch. However if I go to the download/get started page of Pytorch it only mentions AMD and Nvidia.

Now for AMD it requires a card compatible with the ROCm framework. ROCm is AMDs answer to Nvidia’s Cuda framework. I do have a system with a AMD GPU (RX580 with 8 Gb Vram), but unfortunately it is too old for ROCm.

When it comes to price, Intel has the best price in relation to Vram on the card. Intel is followed by AMD with Nvidia still the most expensive. So given that I’m unsure of how well the Marker software will run on Intel, financially the best choice would be AMD. And it would also be the easiest to get working. My experience with AMD GPUs is indeed that they just work.

However I did choose Nvidia. Despite the fact that the chance that drivers could be a hassle and Nvidia being the most expensive. I had a couple of reasons:

• Marker ‘should’ also work on AMD as it is based on Pytorch, the Marker page though only mentions Cuda (Nvidia). So I can’t be entirely sure
• I may do other local AI stuff and Nvidia/Cuda still is ahead of AMD
• Given my long term experience with Linux, including with Nvidia card, I wasn’t too worried about the drivers.

So I chose Nvidia…..

Nvidia driver mess

Installing the card was easy. I started my career in IT in a Computer Shop. So installing a card was nothing new for me. I was replacing an Nvidia card, so I had some hope that it would just work. Spoiler: it did not.

Turns out the 5060 is still considered quite new. The official Nvidia Linux drivers support the card for some time. But unfortunately the combination of recent drivers with new kernels turns out to be quite cumbersome. I tried a fresh install of Linux Mint, which usually supports the Nvidia cards quite well. But I could not get it working. For Nvidia there also open source drivers (Nouveau) which worked well enough, but that leaves you without the Cuda support. I tried older kernels, a download from Nvidia, even Beta drivers. But it just wouldn’t work.

I’ve been meaning to try Fedora for some time as well. The Fedora installation medium however didn’t fully boot, it basically crashed. Now perhaps I could troubleshoot that. But I tried Manjaro first just to see how well that went. Manjaro is based on Arch BTW and straight in the beginning you can choose proprietary drivers. That means in this case the Nvidia provided drivers. And guess what, it worked out of the box with no additional steps.

So for now I’ll stick to Manjaro. Depending on how well that works in the long run I may keep it. I’m sure that in few months the Debian/Ubuntu based distro’s will have fixed the issues. For now it’s Manjaro.

Conclusion

I would love to try out an Intel and AMD card to see how well they work with Marker. It is just a bit too expensive to get some just for testing. Based on the documentation AMD seems like a logical choice, but I cannot from experience tell you if it works. But most likely it will give less of an headache in getting the drivers to work. Based on my experience so far a card with 12Gb should currently be more then enough. But to be future proof on the safe side, a 16 Gb card seems like a good choice to me. Let’s find some other excuse why I needed a 16 Gb Nvidia card :-)