The choice of a star product realization for non-commutative field theory can be regarded as a gauge choice in the space of all equivalent star products. With the goal of having a gauge invariant treatment, we develop tools, such as integration measures and covariant derivatives on this space. The covariant derivative can be expressed in terms of connections in the usual way giving rise to new degrees of freedom for non-commutative theories.