Computationally Efficient Reinforcement Learning Under Partial Observability