It wasn't you; even the mainstream, non-comics viewers wondered where the heck Nick Fury got to during Captain America: Civil War, as the grand experiment he put together in the form of The Avengers collapsed under its own weight and all the members started trying to kill one another.
The writers, though, did have a pretty decent answer.
"Primarily it felt like one too many possible opinions. We didn't want him to take one side or the other, because that's not his place in the universe. And then we didn't want another, 'Is he still with the government? Is he opposed to the government but supporting the government?' It got to be the potential for a lot more polemic discussion that the movie did not have room for," Christopher Markus admitted to the Los Angeles Times. "He's the guy who put it together. He's been the sort of parent figure to the Avengers. Let the parent go away, and see if the kids can handle this. See if the kids can be who they're supposed to be without that governing voice. Um... and they didn't do that good of a job."
...Yeah, you could say that.
Marvel’s “Captain America: Civil War” finds Steve Rogers leading the newly formed team of Avengers in their continued efforts to safeguard humanity. But after another incident involving the Avengers results in collateral damage, political pressure mounts to install a system of accountability, headed by a governing body to oversee and direct the team. The new status quo fractures the Avengers, resulting in two camps—one led by Steve Rogers and his desire for the Avengers to remain free to defend humanity without government interference, and the other following Tony Stark’s surprising decision to support government oversight and accountability.