Available fields and retention depend on the environment. A Googlebot user-agent string alone does not prove that a request came from Google. Check the published IP ranges, or perform a reverse DNS lookup and then a forward lookup of the verified hostname to confirm that it resolves to the original IP.
Group verified crawler requests by URL pattern, status, and date, and compare them with Search Console data. Noindex excludes content from indexing; it does not prevent crawling. Address duplicate URLs and unnecessary crawling while distinguishing the purposes and consequences of canonical URLs, robots.txt, and URL design.
