I don't think we do know. If I had to guess I'd say it wouldn't, since Crowley seems immune to Rowena's magic and devil's trap bullets aren't much more than an annoyance. And somewhere, I can't remember where, I think we learned that Ruby's knife couldn't finish him, either.