What's the point of installing games on my 360's hardrive?

In certain games it can help the loading be more stable and there can be minor speed enhancements in both loading and in-game performance but the main reason I'd say is to help reduce wear and tear on laser and disc reader and to keep down the internal heat of the machine as the hard drive doesn't heat up as much as the disc spinning with a laser reading from it all the time, I'm not certain but I think it also reduces electricity usage for the system when the disc reader is turned off in favor or using the hard drive only.

So in short you'll probably not notice much performance improvements if any at all, but it helps the system do what it does more efficiently.

Personally I always install games I play a lot and plan to play for a while, but if I'm just testing new games or playing something for a short burst and know I won't be playing it again I just play directly from the disc as installation takes a while and puts unnecessary wear and tear on the hard drive instead, it becomes a balancing act on where and how I want to wear out my machine the least.

Another thing is that some games have mandatory installation (either from a separate Install disc or from the main game disc depending on the size of the installation needed) and for those it's usually better to NOT install the game as well as having both the game and the mandatory install files reading at the same time from the hard drive can make load times slower and the game itself less stable if it's a game that loads dynamically during gameplay, like in an open world game.
I think GTAV for example had issues if you installed it because it couldn't read both the mandatory installation data and the game install fast enough simultaneously causing it to crash or something, I don't have GTAV myself so I can't speak from experience but I've heard about others having those issues.

As everyone has stated: reduces loading times, better stability and reduces a lot of noise if you have any pre-slim model (that disc drive spins really fast sometimes).

Also just want to mention that installing games are not always beneficial. Games that were released before the update where you could install them typically run better on the disc vs install. I want to say that I've only heard this and it may not be accurate, as I've never seen or compared it myself.

So wait, this means I literally burned the game onto my hard-drive? Then why do I need the disc to be in the system in order to play it?? It makes no sense.

That's the copy-protection method Microsoft implemented to check that you still have the physical copy of the game, otherwise you could buy a game, install it to the drive and then return the physical copy for a full refund while still being able to play the hard drive installed copy for free.