In a normal scenario, we can always do two Calls like
Cases where CustomerID = Account GUID and Cases where CustomerID = Contact GUID
Somehow, I didn't like this approach of making two calls. Spending sometime helped to reduce a call and I was able to retrieve Cases related to Account and Contact in one call.
Here is the FetchXML I have used and uitype plays a major role here.