You can directly pin apps or games to Taskbar from Microsoft Store. Pin any Microsoft Store apps to the Taskbar in Windows 11.

I was able to create a "Games" shortcut on my Win7 desktop and then pin that to the taskbar following a guide here and it works fine. I click the entry on my taskbar and the "Games" window opens ...