First post, by M-HT
I was looking at a problem with the game Screamer on ARMv8 architecture. It sometimes didn't work, depending on the content of the file CHOICE.DAT.
The problem was with FIST/FISTP instructions.
When storing a value which doesn't fit the target memory, the x87 will store the lowest negative value into the target memory (0x8000 for 16-bit target, 0x80000000 for 32-target and 0x8000000000000000 for 64-bit target).
The current code in Dosbox doesn't do that, at least on ARMv8 architecture - I don't know about other architectures.
I fixed the 16-bit and 32-bit variants by changing these functions in file src/fpu/fpu_instructions.h:
static void FPU_FST_I16(PhysPt addr) {
mem_writew(addr,static_cast<Bit16s>(FROUND(fpu.regs[TOP].d)));
}
static void FPU_FST_I32(PhysPt addr) {
mem_writed(addr,static_cast<Bit32s>(FROUND(fpu.regs[TOP].d)));
}
like this:
static void FPU_FST_I16(PhysPt addr) {
Bit32s value32 = static_cast<Bit32s>(FROUND(fpu.regs[TOP].d));
Bit16s value16 = static_cast<Bit16s>(value32);
mem_writew(addr,(value16==value32)?value32:-32768);
}
static void FPU_FST_I32(PhysPt addr) {
Bit64s value64 = static_cast<Bit64s>(FROUND(fpu.regs[TOP].d));
Bit32s value32 = static_cast<Bit32s>(value64);
mem_writed(addr,(value32==value64)?value32:-2147483648);
}