There is a directory called APK on my tablet. If the device is factory reset, it will offer to install all .apk files in that directory on initial startup.
Is this standard Android 4.0 (ICS) behavior, or something the manufacturer put in?
This is via company, some tablets have a folder called preinstall you can place apk in there and they will be installed after a factory restore, you can remove that folder if you want.