Invalidate properly the cache whenever all_models or app_configs change. This fixes some isolation issues in the test suite.