I submitted a request to Redmine developers to support deleting attachments: http://www.redmine.org/issues/17879