Sunday afternoons there’s an amateur pickup game of ultimate frisbee at a nearby park I play in when I’m not working weekends or otherwise occupied. It’s one of my favorite hobbies. I get good exercise in fresh air and get to compete in a friendly atmosphere. There’s a core group of about a dozen people who come consistently, though it varies week to week.

One week, one of the regular high schoolers invited about ten of his ultimate frisbee teammates from school. Like most teens, they were in their own world. Their group split up between teams, so half of them were playing each other.

The problem was, in their minds they were only playing each other. They were basically goofing off, making silly throws just to impress each other. Meanwhile, the rest of us, those who come regularly to actually play and compete, felt our game dissolve into a chaotic free-for-all. Suddenly the game we’d all come to play wasn’t so fun anymore, and several people ended up leaving much sooner than usual.

This experience popped into my mind as I listened to the (pre-election) story on This American Lifeabout the residents of St. Cloud, Minnesota, who felt alienated and sometimes threatened by the influx of Muslim immigrants to their small town. I suddenly felt a pang of recognition in a complicated political issue I hadn’t given much critical thought.

Setting aside the clear religious differences between the Protestants and Muslims of St. Cloud, the cultural differences alone are strong enough to cause friction. Since whatever culture we’re brought up in carries with it assumptions, parameters, and values that differ from other cultures, anytime a “new” culture arrives with overwhelming force, the change can feel much more disruptive than beneficial, regardless of the benefits it can also bring.

In frisbee, those of us in the usual crowd had a way of playing we all understood and participated in. We knew the “rules” (the amateur versions, anyway) and had played under those rules for a long time. And any week a new person or two joined, the existing culture could easily accommodate them because adding a few new people infuses new energy into the game, adds fresh legs, and hopefully improves the talent on either team. But when it was 10 or 12 new people joining at once instead of one or two, that new energy was so overwhelming that it became its own source of gravity, bending the very fabric of the frisbee continuum.

Not sure how far I can extend the frisbee analogy, but I hope the point is clear. I’m far from anti-immigration. I once was the new guy at frisbee, and am ever grateful for being welcomed into the culture. New people have joined since I’ve been going who have been great additions to the talent pool, and who make Sunday afternoons fun for me.

But we have to face the reality that immigration is hard on everyone, and that the “welcome to America!” romanticism and talent infusion exist alongside the culture clashes, economic strain, and perceived potential for violent religious extremism. Trump has become the avatar for this belief. You don’t have to like him to understand why, nor do you have to support the extreme anti-immigration measures he’s cultivating to acknowledge the legitimacy of the concerns he represents.

It’s a hard issue without an easy remedy, made worse by the Syrian refugee crisis and a clumsy first attempt by the president to do something about it. But I think we can discuss the very real consequences of unchecked immigration—legal or otherwise—without calling its opponents xenophobes and without faulting its proponents for defending the vulnerable immigrants—legal or otherwise—who make this country run.