And if you try to delete them from the emoji keyboard, you'll find that it's just not possible. That requires a trip to the Messages app.