I haven't had a chance to revise what I wrote so proceed with caution hereafter:
I’m by no means an expert on this but I’ve spent a little time researching the subject and haven’t seen anything much better on the internet so far. The best single dissertation that I have seen is in Replica In Scale, Volume 2, No. 3 & 4, Spring and Summer 1974. It doesn’t however, include some of the details below on US Navy bombs.
Some people think that overall yellow denoted practice or inert bombs. According to the Replica In Scale article, Demolition, Incendiary, and Fragmentation bombs were painted dull, flat Yellow during 1920's-1930's. Practice Bombs below 100 pounds were flat white with one two inch blue band; over 100 pounds had a four inch band.
My guess is that bombs were delivered at the beginning of the WWII were still being delivered with a yellow primer coat for corrosion protection and then painted to suit. Here is an August 1943 picture of bombing up a PV-1 in the Aleutians. Note that there is a yellow bomb on the bomb truck. The others seem to be typical of 500 pound bombs - Dark Olive Drab (OD)* with a yellow band around the tail and the nose where the fuse goes, with a thin yellow band in the middle, presumably denoting the center of gravity. (The nose and tail band denoted the explosive in the bomb.) Note that the OD bomb closest to the camera on the bomb cart has a much weathered tail fin box compared to the one behind it. Normally the tail fin box was pristine, since it was shipped in a crate to prevent damage to the fins and ofter stored inside, separately from the bombs. Bombs and box fins from different manufacturers or manufacturing batches might very well be mated in the field, resulting in not only a difference in the "newness" of the bomb and box fin, but in color as well.
Somewhat earlier in the war, US Navy bombs seem to have been painted light gray with no bands, the gray presumably to match the camouflage of the light gray aircraft underside. There is a yellow circle between the mounting lugs, however, which is where the most useful bomb data, e.g. the ammunition lot number, was marked. This picture was taken next to a very early TBF in mid 1942. Note that the fins are oriented at 45 degrees to the usual position. Again, the fins appear to be more pristine than the bomb body.
Another gray bomb, this time for a PBY in January 1943. Note the even more pristine tail fins compared to the rusted? bomb body and the yellow circle on top.
There are several B&W pictures of Doolittle's 500# bombs from early 1942 - They are a light color, presumably gray (since they were probably Navy bombs provided by Hornet) but they could be yellow, and have lots of stencil type markings and mid-section bands.
Looking at the pictures I have so far, it seems that the 500# OD colored bombs had a single thin yellow stripe denoting the center of gravity and the 1000# bombs had a faint red-brown band that corresponded with each of the two suspension lugs instead. This band results from discoloration by the packing band that surrounded the bomb for transportation.
The tail fin orientation depended on the aircraft. Supposedly, the tail fins were generally installed "flat" when the bomb was to be hung externally and "pointed" (rotated 45 degrees so one of the fins aligned with the suspension lugs) when hung from an internal bomb rack. Here are 1000(?)# bombs with vertical fins ready for loading on a B-29; also note the two red-brown bands that are oriented on the lugs as mentioned above and the stenciling between the lugs:
There isn’t much consistency about the fin orientation in pictures so far. In the pictures above, the bomb in front of the TBF has the fins vertical but in instructional pictures of the TBF bomb bay with bombs in it, the fins are “flat”. The fins in the picture of the PV-1 loadout are also oriented "flat" and it has a bomb bay. It's also not clear to me why having the fins vertical was necessary other than a bomb with fins that way might have slightly better directional stability when initially dropped, which if true would be important when dropping them side-by-side and close together so they wouldn't "bump" below the aircraft. Otherwise it seems that having the fins "flat" is an advantage - slightly better ground clearance when on the cart, clearance with the aircraft when installed on a pylon, and better bomb-to-bomb clearance when stacked vertically in a bomb bay.
Here is a good picture of a Mk 43 500# bomb with the vertical fin orientation that also shows the standard markings including stenciling except for that thin yellow band around the middle denoting the center of gravity - you'll note in the picture of the "gray" bombs above that there is a circle of yellow (probably chromate) between the lugs, which is where the production Lot Number goes, probably the only important data on the bomb when it gets to the field.** That's why I think the bombs were painted yellow during production up through the beginning of the war and field painted. Gray would have been the logical color for the Navy since that was the color of the bottom of their aircraft. My guess is that OD from the bomb manufacturing factory became standard with the Army and Navy both using the same color bombs. Why OD? Probably because when they're out in the open prior to being loaded they're not such an obvious target. That might have been important to somebody in the Army and the Navy could lump it and paint their bombs if they wanted.
*You're on your own with respect to what color OD is. At the beginning of the war, it was presumably Dark Olive Drab 41, which was pretty dark. However, the actual shade appears to have varied quite a bit, not only when applied but after only a little exposure in the field. It was replaced later in the war by ANA 613, which by some reports is lighter and either browner or greener, but one wonders whether the bomb manufacturers were told to change in any event.
**The wording aft of the forward lug is something like LOT 1234/EXPLOSIVE/BOMB; The four digit number is representative. The wording forward of the aft lug is FROM PICATINNY ARSENAL; the O's forward of that would be a date or something like P.A. 12-43.