I got this second hand bike that seemingly has a worn out nut/bolt. It’s separate from the angle-adjusting nut, so I don’t know what it does. Should I get this replaced while I still have warranty, or is it something I shouldn’t worry about?

Replace it.

Literally 2 of the last 4 bikes of done repair work on, and im by no means an expert, have had this notch and groove design and the part has been compromised at this spot. One older bike had something like this that allowed the saddle to be angled and a second much newer bike had this on the handle bar. The second bike had probably less that 10 miles ridden on it, and the join was comprimised.