R5900 Emulation

Because I never even try to write meaningful things here anymore, I thought it might be a nice change of pace to actually mention something I've been working on lately. That would be a new R5900 emulator for Noesis. This was kind of born on a tangent from initially just wanting to be able to debug retail PS2 games. More related to the task, though, I've been wanting a way to avoid hand-translating a bunch of disassembled R5900 code every time I run into a new compression method. Which happens a whole hell of a lot. I know that when I'm developing a game and have need to compress some data, my first thought is generally not "I think I will waste hours of development time by making a brand new compression algorithm." Apparently, however, a lot of other developers differ from me in this respect. At least they did, back in the Saturn-PS2 eras. I suppose good platform compression standards were a tiny bit harder to come by back then, though. These days everyone seems to just be lazy and use zlib. Hell, why not? It's usually good enough.

So then I thought, wouldn't it be great if instead of having to disassemble code and translate it to extract encrypted/compressed data, I could just run the actual native decompression code in Noesis, after doing a little bit of work to figure out what the function expects as stack/register inputs. As a bonus, tons of PS2 ELF binaries leave their symbol table fully intact. So in a good number of cases, I can execute my code from symbol offsets, likely retaining compatibility with different versions of the same game from different regions and so on.

On top of that, I really like writing CPU emulators. Sometimes. Once I start getting into it and writing logic for yet another variant on a bit shift instruction, it becomes less fun. Nonetheless, though, I found the idea of writing a R5900 emulator really appealing. I figured it would also improve my MIPS-reading skills, which it has. My MIPS was pretty rusty before I started the emulator.

Bujingai was the first to fall victim to my emulator of ruin and decompression. I was immediately able to find the game's decompression routines through the ELF symbol names. It then only took a few minutes to figure out what should be on the stack and how to set the registers before executing the routine, and the output would be in the R5900's memory right where I pointed one of the values fed in via register. Using data breakpoints in my emulator made figuring all of that out pretty trivial.

Bujingai also stores ROM tables for each of its asset containers within the executable itself. These are even attached to appropriately-named symbols, so the info table can be retrieved automatically based on the name of the file being extracted. All of this basically turns into some code like this in Noesis:

	BYTE *elfData = Noesis_LoadPairedFile("Bujingai US ELF (SLUS_208.95)", ".95",
				elfDataSize, NULL);
	DWORD sysMemSize = 0x02000000;
	sysContext_t *ctx = R5900_AllocSysContext(sysMemSize);
	bool loadedElf = R5900_LoadELF(elfData, elfDataSize, ctx);
	unpooled_free(elfData); //no longer needed once it's been loaded
	if (!loadedElf)
	{
		richprintf("ERROR: Unable to load ELF.\n");
		R5900_FreeSysContext(ctx);
		return false;
	}

	//try to base offsets on symbols if possible
	elfSymbol_t *decompSym = R5900_FindSymbol(ctx, "decomp", true);
	elfSymbol_t *decompGetByteSym = R5900_FindSymbol(ctx, "decompGetByte", true);
	DWORD codeOfsDecomp = (decompSym) ? decompSym->val+0x40 : 0x00125140;
	DWORD codeOfsHalt = (decompSym) ? decompSym->val+0x9C : 0x0012519C;
	DWORD codeOfsReadEv = (decompGetByteSym) ? decompGetByteSym->val+0x0C :
						0x001250EC;

	//stop the simulation at this offset using a custom instruction
	R5900_MakeInstruction(ctx, codeOfsHalt, OP_CUST_STOPEXEC);
	//v0 is guaranteed to be the source offset here. the readevent function sets
	//the source buffer pointer for local tracking.
	R5900_SetLogicHook(ctx, BujingaiBin_ReadEvent, codeOfsReadEv);
	ctx->cpu.regs[REG_SP].ur[0] = 0x01000000; //put the stack here

	//finds the symbol pointing to the file table for this container
	elfSymbol_t *rlSym = R5900_FindSymbol(ctx, romTableSymName, false);
	if (rlSym && rlSym->size >= sizeof(bujinRomInfo_t))
	{
		romInfo = (bujinRomInfo_t *)(ctx->sysMem+rlSym->val);
		romInfoNum = rlSym->size / sizeof(bujinRomInfo_t);
		richprintf("Found romInfo table for '%s'!\n", romTableSymName);
	}
	else
	{
		richprintf("WARNING: Couldn't find symbol, bruteforcing.\n");
	}

	DWORD dataOfs = 0x01800000; //stick the compressed file here
	memcpy(ctx->sysMem+dataOfs, fileBuffer, bufferLen); //put it in system memory

	//...
	//loop through all of the rom table entries and do this
		ctx->cpu.pc = codeOfsDecomp; //start of decompression code

		//s0/s1 will be used in looking for pointers around here
		DWORD *x = (DWORD *)(ctx->sysMem + dataOfs + bufferLen);
		DWORD decompDestOfs = dataOfs + bufferLen + 0x00100000;
		x[1] = decompDestOfs;
		x[2] = dataOfs + cmpOfs; //start of the data at the compression header

		//make s0 and s1 point to x
		ctx->cpu.regs[REG_S0].ui[0] = dataOfs + bufferLen;
		ctx->cpu.regs[REG_S1].ui[0] = dataOfs + bufferLen;

		BYTE *decompChunk = ctx->sysMem + decompDestOfs; //output dest
		R5900_RunCPUForCycles(ctx, -1);
		DWORD decompChunkSize = (ri) ? ri->decompSize :
							ctx->cpu.regs[REG_V1].ui[0];
		if (!romInfo)
		{ //couldn't find rom info, so just guessing
			DWORD readSourceBytes = g_lastBujRead-dataOfs;
			cmpOfs = readSourceBytes;
			int tryReadUp = 0;
			while (cmpOfs < bufferLen-4 && tryReadUp < 16 &&
				memcmp(ctx->sysMem+dataOfs+cmpOfs, "CMP3", 4))
			{ //read up, since there is no consistent alignment
				cmpOfs++;
				tryReadUp++;
			}
		}

		int outFlags = (ri) ? ri->flags[1] : -2;
		BujingaiBin_AddFileForOutput(decompChunk, dstTotalOfs,
			decompChunkSize, outFlags, bitStream, dstOfsList);

So that covers it. No translating disassembly and writing custom decompression routines. The decompression happens pretty quickly, too, all things considered. I also plan to expose the R5900 core to plugin authors, once the interface is more stable and I've implemented a few more games to put my instruction logic to the test. I did write about 50 instructions while drunk over St. Patrick's day, so I was surprised Bujingai's decompression routine actually managed to run correctly on the first attempt.

I'm sure Señor Casaroja will be very pleased to get this new functionality in his amazing Noesis.

Comments

17 comments in total.
Post a comment

ukurereh

April 9, 2011 at 3:11 pm (CST)
@Hung : http://forum.xentax.com/viewtopic.php?p=50037#p50037 Posts on this page have some good info...

Including...
"In the lastest version [of GSdx] you can simply press Shift+F8 to dump the current scene, along with a normal snapshot u'll get an .obj & a .gs dump."

ukurereh

April 9, 2011 at 2:59 pm (CST)
@Hung : Another option is GLIntercept and an OpenGL graphics plugin (not sure if it exists, I'm too lazy to check). No promises, but I wish you luck.

@Rich Whitehouse : Out of curiosity, which PS2 debugging/disasm tools do you use? I know ps2dis has experimental r5900, though I don't know if it's usable... and Project Artemis by GSHI was supposed to be a native debugger for PS2, but it seems dead at the moment.

Hung

April 8, 2011 at 11:29 pm (CST)
@ukurereh : still not complete rip with 3DVIA.You just get a flat or skew mesh and need to modify, i'm still searching for a good rip.

ukurereh

April 6, 2011 at 11:00 am (CST)
@Hung: It's already possible to rip stuff from PCSX2. Try searching google for |rip pcsx2 models|.

I haven't tried it, but 3DVIA Printscreen might help: http://www.3dvia.com/products/3dvia-printscreen/

Hung

April 3, 2011 at 11:04 am (CST)
May be this's not relate to what you say but can you coding a plugin that rip model from pcsx2.Sorry for my bad english too

Rich Whitehouse

March 23, 2011 at 12:59 am (CST)
Yeah, I just recently decided to start digging through some of my old PS2 games. It's been pretty interesting. I do really wish I had a PS2 emulator with a fully functioning debugger, though. PCSX2's debugger is horribly broken, and my emulator is a pure CPU emulator, so it's nice for running isolated code segments, but useless for running the game on its natural execution path. Something that also really needs to be added for PS2 debugging to be practical is breakpoint hooks for the dynarec code, so that the emulator automatically goes to interpreted mode when it hits the general code page where your breakpoint has been set. Unless you want to debug your emulator on the native platform and step through setting breakpoints in your dynarec pages yourself and using conditional breakpoints in your mem read/write functions as ghetto R5900 data breakpoints. (or having to figure out the relative address of the R5900 address in your process space each time, I suppose) But, to hell with that.

I've talked with revelation (from Xentax) before about the possibility of making an IDA Pro debugging plugin for PCSX2, which I think would be the ideal solution. Hopefully one of us will end up taking the initiative there eventually. I've been meaning to look at the plugin SDK, but I've let myself be distracted continually.

I don't think Casaroja intends to split the program out, although he does occasionally have passing thoughts about integrating better archive support, with standard interfaces for browsing packages and extracting and/or adding individual files. He will probably continue to be too lazy to do that for a while, though.

ukurereh

March 23, 2011 at 3:54 am (CST)
Holy flapping jeebus what. Never thought I'd see MIPS stuff popping up here. I know I try to go back and read MIPS disasm stuff every few months or so and fail horribly. So maybe this will finally put encrypted .CVM files within reach.

It looks like Señor Casaroja pursues the dual goals of mesh file stuff and container file stuff. Do you think he might split the-program-which-must-not-be-named into 2 separate programs at some point?


Comment Pages:
Previous ... 1 [2]

Post a comment

Name:


Enter the following (refresh if you can't read it):



Comment:

Stream Tags

Projects

Everything

Personal

Analysis

Indie

Technology

AVALANCHE

R-Cube Engine

Modding

Game Design

Commercial

iPhone

Delight

Recent Comments