Angus MacDonald says he's concerned those asylum seekers housed at the army installation may congregate in Inverness.