Place ESL face-down on a flat surface OR using your hands hold the ESL around the bezel avoiding applying pressure on the screen.
On the sides, there are two latches that, holds the cover in place.

Use a tool like a small screwdriver to carefully pry open the latches and remove the cover.

Once opened you can replace the battery with a new. Be careful not to break the small pins when attaching the cover again.

When you place the new battery in the display it important to place it in the right direction. if you turn the battery the wrong way the display won't work.
Red = positive
Blue = negative

You can identify the positive(Red) by the 3 dots on the copper field. On the battery, it's the connector that is attached to the top that is positive.

You can manually check that display is working by force updating the display.
If you don't force update the display it is no problem. The display will update the next time the system has an update for the display.