How much damage do you usually take? Have you tried isolating the logic to just importing keras and loading your model on a very basic algo to make sure this is the root issue? I haven’t worked with keras before, but is the model you are loading unusually large or something of that nature?
I believe people have used keras before without this issue, so I am trying to diagnose if there is anything going on specific to your case