Does it void my warranty if I use Rockbox?
Yes.
However, most large companies tend to be lenient with warranty claims. So even if using Rockbox will technically void your warranty, quite probably the aftersale service won't cause any trouble. Of course, they can, but it is not really in their "commercial interest" to do so, and there is plenty of testimony from users who got their device replaced even if the issue was caused by misuse (example: hard drive dead due to a fall, or even LCD screen broken).
Iriver now have the following statement in their online warranty terms, specifically naming Rockbox as voiding the warranty: "This warranty is void if [...] the player is opened, repaired, modified or altered by third parties e.g. Rockbox, unless such repair, modification or alteration is expressly authorized in writing by iriver". Archos has also been known to refuse warranty repairs for people that have returned their units with Rockbox flashed.
Can the changes or the software that Rockbox suggests or offers possibly damage my MP3 player?
All firmware mods that are presented are still highly experimental. Use them at your own risk. We offer no guarantees that this software, or the hardware modifications we show, will not damage your player or void your warranty. That said, we have not been able to damage any of our units by modifying only the firmware. On the Archos, you can accidentally password protect your hard disk, but there are ways around that. (See ArchosFAQ)
Note: Several companies, including Archos and iriver, claim installing Rockbox voids their warranty.