Navigation algorithms and the corresponding observability analysis for formation flying missions are developed. The methodology of the observability analysis relates the physical geometry of the observers, as well as the spacecraft formation, to several measures of system observability. Relationships between these observability measures and the state error covariance are then derived to provided estimated bounds or forecasts for the expected navigation accuracy. These methods range from conservative time-invariant analytic bounds to more representative numerical forecasts using common dilution of precision metrics. The research also examines the robustness of the extended Kalman filter when simultaneously processing inertial and relative range measurements. It has been shown that processing relative range measurements in conjunction with inertial range measurements can directly increase the accuracy of the inertial state estimate. However, it has also been shown that when there is relatively large uncertainty in the state estimate the addition of relative measurements can cause an otherwise convergent filter to diverge. This dissertation considers several methods for preventing this divergence, as well as an in-depth examination of second-order terms to explain the basis of the problem. In particular, to illustrate their potential significance, analytical bounds are derived for the second-order terms.