Here’s a challenge for wiser RSS-wranglers than I: as aggregation becomes a more widely used tool for populating web sites, how do we prevent RSS feeds from being cluttered with multiple identical posts?

I was just looking at the Technorati tag page for net2, where a couple of my Net2-related posts have each appeared twice. That’s because my blog is being aggregated in full at Web of Blogs, an aggregation set up for the Web of Change conference. Once I get my Social Signal aggregator up and running, that may provide another duplicate of many of my blog posts.

It would be great if Technorati, Feedster & other searches could recognize true duplicate posts, and only show them once — or if there were a way to strip duplicate posts out of a feed when aggregating onto another site (for example, NetSquared’s aggregator page.)