Trying to extract from Firmware

[OffWorld]

I decided to wade into the deep end of the pool and start learning how to extract from firmware and eventually to cook a rom.

First I needed the right tools:
https://sites.google.com/site/naobsd/android-tablet/hsg-x5a

I have no problem building the Telechips Utilties. That's as simple as extracting the folder, cd-ing into it, and running this in my X11 terminal:

#gcc -o tccpack tccpack.c
#gcc -o tccunpack tccunpack.c
#gcc -o tccsplash tccsplash.c

I create a directory to extract stuff to and cd into it:

#mkdir ~/extract && cd ~/extract

Then I can successfully unpack the nand image that comes with the firmware download using:

#~/tccutils/tccunpack /downloadedfirmware/tcc8900_mtd.img

That gives me the following files in my "extract" folder:
system.img
recovery.img
boot.img

But the tccunpack is apparently NOT for further extracting anything, so I need "unyaffs" for that. There is a precompiled binary for Windows, but those of us on Mac and Linux will need to build it ourselves. Which is fine since it apparently needs to be patched to deal with Telechips images anyway.

So I get the unyaffs files and the patch. Drop those in a folder called "unyaffs" and do the following in a terminal:

#patch --dry-run -p1 -i unyaffs.c.diff
#File to patch: unyaffs.c
#patching file unyaffs.c
#patch -p1 -i unyaffs.c.diff
#File to patch: unyaffs.c
#patching file unyaffs.c

(obviously I do a dry-run before I actually patch)

Usage is pretty simple:

#~/unyaffs/unyaffs ~/extract/system.img
#end of image

Now this is where things fail. If I run this on a "system.img" fulled from the SDK my "extract" folder gets filled with the usual Android system folders. But any "system.img" from a Telechips firmware produces NOTHING. No folders, not even an error.

This is only for the "system.img" - I haven't bothered with "boot.img" or "recovery.img" since they apparently need additional tools to extract them, and for now I'm only interested in the system files.

I should mention that I first tried vanilla unyaffs and that would give me "segmentation fault" errors on the Telechips images, but also extracted the SDK images just fine.

I suppose if I can't extract the folders directly a nice option would be if I could use the "system.img" in the Android Emulator, which I recall reading somewhere is possible to do, but I don't remember how.

Any ideas?

 

[wintermute]

That steps you've done sounds right. When I run unyaffs on system.img, the contents of system gets unpacked (/apps, /bin, /etc...). I've done it for the 11/16 rom as well as one of the 2.2 ones. The files are extracted on the same level as the system.img file, i.e. not in a subdirectory.

Have you tried running unyaffs on boot.img and recovery.img? In both cases you will just extract the kernel.

Also, have you tried other firmware? Maybe the particular one you tried is corrupted somehow.

 

[wintermute]

Oops, first, a correction to my previous post. To extract boot and recovery, the program to use is split_bootimg.pl, not unyaffs. I had forgotten that.

Secondly, unyaffs gives seg fault if the page size of unyaffs and system.img does not match. fun explains here:

Haipad M701-R [HY] firmware cannot be extracted ? - SlateDroid.com

So I think the current patch in naobsd's site is set to work for the original M701_2G images. Change page size to 8192B/256B instead for the 4G HY model. I'll try this myself to see if it works - right now my compiled unyaffs seg faults on the HY images.

 

[Loonix]

yes, that is right.
You need one unyaffs version for the 2GB firmwares and another for the HY firmwares.

Have a look at my thread on slatedroid, maybe you can figure out why i never managed to re-pack my unpacked firmware images:
unpacking and packing tcc8900_mtd.img (not fully working) - SlateDroid.com

 

[OffWorld]

You're right, I was trying it on some of the older firmwares (the 9/16, 9/19, etc.) and it failed. Guess they are corrupt. Worked fine on the 11/16 though.
As I mentioned in the group thread about recovery I've used split_bootimg.pl to extract the boot.img and recovery.img (stock and CWM) and I'm still not sure WHY the CWM that is available doesn't work.

I'll look into the unyaffs page size, I suppose it's possible since the 2GB Gingerbread is also supposed to work for 4 GB Micron it might be seg faulting for the same reason as it does on the 4 GB HY.

 

[Loonix]

my guess is that the boot.img needs to have an option to boot into a recovery mode and that that option was removed with the upgrade to 2.3
It could also be possible that since i belive the boot.img contains the kernel its instructions are simply not compatible with the recovery that should base on 2.1

 

[OffWorld]

So, let me see if I can understand this in terms I'm more familiar with.

The boot.img has the main kernel in it, the same one used when you boot normally.
The recovery.img has it's own kernel, independent of the other one.
In theory they could be different versions, just like having two different operating systems installed in different partitions on your hard drive?
But boot.img has to have the option in it to boot to the recovery partition?
I guess if I were thinking of the boot loader as being like grub, bootcamp, or Windows Boot Manager you're saying it doesn't have an entry in it for the recovery partition?
Is it possible to send line commands to the boot loader, or edit a boot options file like you can with most boot loaders?
(sorry lots of questions)

 

[fun]

sorry I don't read your posts well so here is general tips

boot.img is for normal boot and recovery.img is for recovery boot. both have same structure (kernel+ramdisk) and works independently. (may differ on other device)
you can use different kernel for boot and for recovery. but kernel depends on bootloader(lk.rom) version. you need to use kernel which is compatible with flashed lk.rom. (generally kernel and lk.rom from same firmware release should be used)

in addition to system.img(yaffs2 image), pagesize is also important for boot/recovery image. see split_bootimg.pl output carefully. HY model uses 8192B pagesize and non HY model uses 4096B pagesize.
you need to specify correct pagesize for mkbootimg --pagesize.

I'm using unpackbootimg instead of split_bootimg.pl because it makes files which contain these parameters.
my site is not updated well. sorry for my lazyness.

 

[wintermute]

Thanks for the info fun.

So to get new compatible clockworkmod recovery, one has to rebuild clockworkmod using the kernel extracted from 2.3 recovery.img?

Another question, is the system partition independent from the boot kernel? That is, would it be possible to run a combination of boot.img and lk.rom from 2.1 + system.img from 2.2 or 2.3? If this is possible then the old clockworkmod recovery should work right?

Thanks!

 

[fun]

So to get new compatible clockworkmod recovery, one has to rebuild clockworkmod using the kernel extracted from 2.3 recovery.img?

generally yes

Another question, is the system partition independent from the boot kernel? That is, would it be possible to run a combination of boot.img and lk.rom from 2.1 + system.img from 2.2 or 2.3? If this is possible then the old clockworkmod recovery should work right?

yes and no.

e.g. I made CM7(2.3) with 2.1 kernel. but I can't use libs from 2.3 system for my CM7 with 2.1 kernel because it uses some new interface which exist only on 2.3 kernel.

you shouldn't mix 2.1 ramdisk(in boot.img) and 2.2/2.3 system. ramdisk is a part of userland and system is rest of it.
kernel module may exist in ramdisk and/or system. it needs to be synced with kernel.

recovery is independent from boot/system on tcc8902 devices. old recovery may work with new boot/system on some case. but lk.rom and kernel(in boot.img and in recovery.img) need to be synced. if new lk.rom is incompatible with old kernel, old recovery with old kernel will not work.