If you are planning to remove a web part from a page, ALWAYS click on Delete menu rather than pressing the X button. Clicking on X button would close the web part and you will receive a warning in your servers event log. The web part is not actually deleted from the page, it is removed from the Visible area of the page and placed in the Web Part Page Gallery.
You can delete a CLOSED web part by using the Web Part Maintenance page. Browse to http://www.yourserver.com/yoursite/yourpage.aspx?contents=1
You could see the web part that has been closed. Now, delete it by selecting the checkbox and pressing Delete
