I will answer your question by first quoting your questions you asked in explanation.
Is it still hold the place in the memory?
Yes, as soon as the activity is live either it is in foreground or not, its memory is still stable. Nothing got removed from the device memory until you finish the activity by yourself when moving from that activity.
Android OS temporary removes it and recovers it back when the user navigate back?
No, there is not such thing in Android OS which will remove the activity once its not in foreground and recovers it back when you go back to that activity.
Should i manually manage the memory in that case?
It is possible but it is complicated process to implement such system in your android app and even to achieve this thing you would have to go for extra overhead when the activity to which you are going back will become slow because of course you will get the images, videos etc back at the beginning of the activity when the activity is not on the foreground, so it can slow down the performance at that point.
Its better to leave it on the Android OS, i am agree Android OS is not human but it knows well than the human beings, but the most important thing you should also consider is you should optimize your script, use less resource consuming components like RecyclerView
etc, for example. Its my suggestions now its up to you.
Almost same question is also asked on SOF, you can read it here:
Android memory management in activity lifecycle