Post some pictures of your CPU heatsink/cooler, if possible.
If the heatsink is large enough, you can slow down the original fan via either 3-pin resistor adapters or direct "hot-wiring" of the fan for 5V or 7V operation. I did the latter (7V operation) on my Pentium III 933 MHz CPU, simply by taking the negative (black) wire out of the fan connector and wiring it to the 5V rail. This gives a difference of 12-5 = 7V going to the fan. Just beware that doing this will make the fan tachometer (RPM) not visible to the motherboard anymore. On the other hand, wiring the fan directly to 5V (via connecting the red wire to 5V and leaving the black wire normal) won't have this issue.
Then again, if the original heatsink fan is loud due to worn out bearings, then you should replace it indeed. Most socket 370 coolers came with 50 mm fans and a few with 60 mm fans. 70 mm fans are rare, but not non-existent. On that note, and depending on how handy you are, you can also go with the option to adapt a 70 or 80 mm fan to your current cooler via 3D printing or just cobbling something together out of common materials. I've done the latter many times. It never won any beauty contests, but is effective and allows me to use my large stock of 80 mm fans... and at a lower voltage for the same airflow, making the systems nearly silent.
As for the PSU, perhaps also post a picture of it, or rather its label. Or at least tell us the brand and model #. Some cheap PSUs from that era wired their fans directly to 12V, so always sounded loud. Normally cheap PSUs like that are not worth re-using (they tend to suffer from bad caps, among other issues), but a few worthwhile ones do come up every now and then that are worthy of a recap (and fan silencing 😉 ).
In any case, don't replace your original PSU with a modern one just quite yet. I see this advice given here very often, but depending on which part of the world you live, or in particular what brands of PSUs are commonly available there, you might be in for more trouble with the new PSUs. If you go with a decent brand like EVGA, Antec, Corsair and similar and a model that has a 5V rail rating capable of at least 15 Amps, that would be fine. But if you go with a modern "no-name" PSU, it's probably going to be a 12V-based group regulated design that will likely not play well with your PIII PC. So just be aware of that.
*EDIT*
Ah, you're ahead of me already and posted pictures of the system - cool! 😀
Yep, that P3 cooler looks large enough that you can just slow down the fan via voltage mods to make it more quite. Try the "7V" mod first (remove black wire from fan connector and wire it to a red wire on a molex connector or similar.) Should still keep the CPU plenty cool and won't sound like a jet engine anymore. Even a 5V mod might be OK with this fan, being a 70 mm fan. But before jumping to that, see how hot the CPU gets with the 7V mod first. I'd be surprised if it goes above 40C under full load with this cooler. Pentium 3 CPUs top out at around 35-ish Watts, so they are not exactly hard to cool.
As for the power supply... I see it's a Q-tec, which I don't recall running into off top of my head. But it's a generic brand probably made by one of the less-than-stellar cheapo OEMs like Deer/Solytech, Codegen, Linkworld, and etc. A picture of it with its top cover removed (disconnect from wall AC power before doing this [!!!] ) should tell us more if it's OK to re-use or not... tough I suspect even if it is, it will probably need a recap already or at some future point not too long from now.
Finally, in regards to video cards...
I think a Radeon 9600 or 9550 (128-bit memory bus version) would be a pretty sensible step-up and likely not too expensive or hard to find in your country. Avoid stepping up to Radeon 9700 or 9800 video cards, though - these run hot and rarely last long... not to mention they draw a lot of power from the 5V rail, which will make finding a new PSU (if it comes to that) even harder.
Alternatively, if you prefer to step into the nVidia side of things, a GeForce 4200TI or an FX5600 would also be a good fit for this system. Only problem with these is that they have become a little more expensive and not as easy to find as the Radeon 9550/9600.
Going beyond these, you'll be mostly getting bottlenecked by the CPU, so probably not worth it. But if sticking with Windows XP, a cheaper GeForce 6600LE (instead of a GT) could also be used.