Django is very modular so you can pull out or change the bits you do not like (templating you can use jinja, orm you can exchange for sqlalchmey).
Flask, you build what you want (enough said).
Flask is an awesome option, I'm building things with it for some years now.
If you are pulling out the Django templates and the ORM you should just use Flask. You shouldn't choose a framework and then immediately start taking it apart if there is an alternative that comes the way you want basically out of the box.
Django is very modular so you can pull out or change the bits you do not like (templating you can use jinja, orm you can exchange for sqlalchmey).
Flask, you build what you want (enough said).