The Hubble Ultra Deep Field (HUDF) contains a significant number of B-, V-, and i'-band dropout objects, many of which were recently confirmed to be young star-forming galaxies at z similar or equal to 4-6. These galaxies are too faint individually to accurately measure their radial surface-brightness profiles. Their average light profiles are potentially of great interest, since they may contain clues to the time since the onset of significant galaxy assembly. We separately co-add V-, i'-, and z'-band HUDF images of sets of z similar or equal to 4, 5, and 6 objects, pre-selected to have nearly identical compact sizes and the roundest shapes. From these stacked images, we are able to study the average(d) radial structure of these objects at much higher signal-to-noise ratio than possible for an individual faint object. Here, we explore the reliability and usefulness of a stacking technique of compact objects at z similar or equal to 4-6 in the HUDE Our results are: (1) image stacking provides reliable and reproducible average surface-brightness profiles; (2) the shape of the average surface-brightness profile shows that even the faintest z similar or equal to 4-6 objects are resolved; and (3) if late-type galaxies dominate the population of galaxies at z similar or equal to 4-6, as previous Hubble Space Telescope studies have shown for z less than or similar to 4, then limits to dynamical age estimates for these galaxies from their profile shapes are comparable with the spectral energy distribution ages obtained from the broadband colors. We also present accurate measurements of the sky background in the HUDF and its associated 1 sigma uncertainties.