Let us do all that work for you and just lean back...
Synchronize Microsoft 365 GAL group members
Now only Microsoft 365 users who are members of a specific group can be synchronized as contacts. This feature is available for all newly created connections.