I've written before that we should only estimate if having the estimate will change someone's actions. Normally, this means that the team should estimate work only if having the estimate will either:
enable the product owner to better prioritize the product backlog, or
let the product owner answer questions about when some amount of functionality can be available.
But, even if we only ask for estimates at these times, there will be some work...