Yes, that’s basically the way of doing it. However, in your case you would only see that on an actual Page, but note that I mean “page” in the sense of a WordPress Page here — not a website page.
(A WP page is a static page, created in the PAGE section in the WP admin. See the link above for details).
If you want your div to show up on every single page in your website, you’d be better off adding the div in HEADER.PHP.