Perhaps shaky cam is not dissimilar to the way Van Gogh's art was viewed when he was alive. As mentioned in the film people tended to mistake his style for clumsiness and ineptitude. And you are far from alone in believing that shaky cam is used to show what humans see. While you are correct to an extent, it is used that way sometimes. It is far from the only way it is used and I don't see it as used for this purpose in At Eternity's Gate.

As Van Gogh was clearly a mentally unwell man the unsteady camera is used to give a sense of his state of mind. Notice how when he becomes confused or distraught the camera bounces about a lot more. The more at peace he is the stiller the camera is and the smoother the motion. Remember that film tends to be used to give a sense of something, a feeling or mood and not to necessarily recreate reality exactly as it is.

And I believe the hand held style here is used to create a mood of instability and unease. If it makes you feel uncomfortable that is probably the point.