If we find living organisms on another planet, with no evidence of precursors of any kind, then we would suspect that the organisms were brought there from somewhere else. And the involvement of intelligence would be a strong explanation in that case.
In your thought experiment, it doesn’t matter that the objects of interest are bacteria. Anything that is anomalous would require similar explanations. So if we found a chuck of obsidian on planet that had no obsidian anywhere else, we would be justified in postulating that the obsidian fell from the sky. And so on.