Charge-orbital ordering is commonly present in complex transition metal oxides and offers interesting opportunities for novel electronic devices. In this work, we demonstrate for the first time that the magnetization states of the spin valve can be directly manipulated by charge-orbital ordering. We investigate the interlayer exchange coupling (IEC) between two epitaxial magnetite layers separated by a nonmagnetic epitaxial MgO dielectric. We find that the state of the charge-orbital ordering in magnetite defines the strength, and even the sign of the IEC. First-principles calculations further show that the charge-orbital ordering modifies the spin polarized electronic states at the Fe3O4/MgO interfaces and results in a sufficiently large phase shift of wave function which are responsible for the observed IEC sign change across Verwey temperature. Our findings may open new interesting avenues for the electric field control of the magnetization states of spin valves via charge-orbital ordering driven IEC sign change.