If you long press on the icon, a garbage can should appear or you'll see a list of options and deleting would be one of them. If the garbage can appears, drag the icon to it and drop it in.
What Spider described will just remove it from the homescreen.
To uninstall it, go to Settings - Apps - (Manage) - All
Find the app you want to get rid of and tap the uninstall button.
If it is a a stock (pre-installed app) you will not be able to uninstall it. If your tablet is running Android 4.0 or higher you can Disable the app (or possibly uininstall updates then disable)