Ability To Visit Grandfather Staying At My Uncle's Home

There is a situation in my family wherein my grandfather (mother's father) is staying at my Uncle's(mother's brother) home. There is much friction between our families and are unable to visit. Please advise if it is required to have permission from my Uncle to visit my grandfather at his home. Will it possible for them to sue us in case we enter the home without their actual consent. Please advise as this is very crucial for my help.