HDMI is a digital video/audio connector used for HD signals. It's similar to DVI (in fact, the video portion is exactly the same, and you can buy cables and connectors to convert between the two) but with audio. Unlike component output, HDMI is digital in nature and thus theoretically has a better picture (you can skip the digital-to-analog conversion).
The big controversy over Sony dropping HDMI from the cheaper PS3 (which they've since recanted) is due to copyright protection for Blu-Ray videos. Studios and set a specific flag on the movie (the ICT flag) that will make the movie require an HDMI connection with HDCP in order to play at higher resolutions (720p, 1080i, 1080p). Without HDMI, you'll be limited to current DVD resolutions, which totally negates the purpose of an HD video format like Blu-Ray. While studios have said they don't plan to ship movies with the ICT flag set until 2011, there's nothing stopping them from doing so. As well, Sony claims that the PS3 will have a 10 year lifespan, which means that it'll still have 5 years left when the ICT flag starts getting set on movies, making the cheaper PS3 effectively useless (which is why they stepped back and decided to re-add HDMI).
Most people who are against HDMI hold that opinion for two reasons:
1. They dislike DRM (digital rights management) and copy protection on principle, so they object to HDCP.
2. Recent churn in the HDMI specification has caused older sets with HDMI to become useless (HDMI does not require that HDCP be implemented, for example, so an older set with HDMI but no HDCP is no better than using component video). With HDTV adoption on the rise, it's no longer bleeding edge early adopters who are getting screwed when these specifications change.