Solar panels have a "power curve, voltage vs current at a specific "insolation".
In the most general sense, and very approximate, the "max power point" ( max total power ) will occur at about one-half of the Max open circuit voltage. ( at zero current ). That figure is close enough to get a starting point, but each panel is different. ( If it were easy, every one would be doing it!

)
There are various kinds of controllers, but in general terms, they attempt to convert the widely ranging voltage ( and current availability at a particular sun level ) to a voltage that is appropriate for the current charge state of your battery, while attempting to achieve the maximum power output of the panel. It is surprising ( to me, at least ) how complex this can get, in the details. Thankfully, in practice, it is much simpler; someone has already done the work, and there are many not particularly expensive controllers available.
With a very small solar panel, vs a relatively large battery, the controller is not necessary, but once the battery is charged, the panel will still produce power. As the battery begins rejecting further charge, the current drops, and panel voltage rises. at full sun, you will approach Vmax and even at very low outputs there will be current flow. Which will be dissipated as heat, at the battery. A little heat is unlikely to cause issues, but since a controller will stop this, I consider a controller to almost always be a plus.
Have fun with it!