Let us imagine a tiny congregation at some rural crossroads. Regular attendance is down to five or six elderly ladies (and a neighborhood cat that sometimes snoozes in the open window). Is this a real church?
The tiny congregation gathers for worship. They lift up their prayers. The pastor celebrates communion and all receive. Those are all marks of a real church.
The members pray for each other, confide in each other, "watch over each other in love." Those are also marks of a real church.
The only other time the building is usually open is when the same ladies gather on Wednesday afternoons to do quilting and other needlecraft. Selling home-made quilts helps keep the doors open, but they also give away many smaller pieces -- to preemies born at the local hospital, and veterans in the local nursing home. The ladies also pray. A lot. They take the time to pray for dozens of people, by name, who have been referred to them. They pray for missionaries in various countries, too, many by name. They send donations to those missions, too. These ministries to the needy in the community and to the church around the world are real ministries, being done by a real church.
If this tiny cell of Christianity seems less than fully a church to you, perhaps you need to examine your own assumptions. Yes, they can't pay a preacher much, but that has little to do with the overall value of their life together and their service to God. There were many such small conventual houses in the Middle Ages, where a priest-confessor came in on a regular basis, and otherwise left the sisters to their work of study, love, and prayer. By any reasoning, this is a real church, and it deserves our support.
So, when do you know it's time to close the doors and move on? I think there are two occasions. One possibility is, the small congregation can no longer keep up its own existence, particularly its meeting place. Everything they can scrape together keeps the lights and heat on, but repairs fall ever further behind. The needs of the building and grounds are distracting from the mission of the church. (The same could be said of many a large and busy congregation, which struggles to meet its mortgage and does fund-raisers all the time and only gives prayer and teaching the faith a lick and a promise.)
The other way you know it's time to move on is when an unrealistic nostalgia or a burdensome guilt is the motivating factor of the common life. If people are happy with what they are, that's one thing; but if they are holding out hope for a revival that will make them what they used to be, that may be a waste of energies that should be devoted to God in a different manner. Likewise, if they're only keeping going out of a sense that they don't want to be the ones who "gave up" and disappointed all the saints that went before, well, that's too big a burden for anyone to carry. All your ministry attempts are just making you miserable, rather than bringing you peace. (Again, there are some larger churches with considerable salaries that do these things, too.)
The fact that it is tiring and disappointing for clergy to organize pastoral care and sacramental ministry for these tiny outlets is NOT a good enough reason to abandon them. We who have been ordained to the ministry of Word, Sacrament, and Order are servants of Christ, not CEOs. We go where we're sent, and we make time for people according to a standard different from the world's.
Yes, there's a time to call it quits. And there's a time to embrace a different future, even if you could keep doing what you've always done. But that's for each community of faith to decide. We clergy need to undertake to cover our responsibilities efficiently, which may mean we don't try to do the full pastoral and programmatic shtick in every location. And in our strategic planning, we don't try to launch congregations that will average less than a certain number. But we accept aging and death along with growth and life, for congregations as well as individuals, and we try to walk with each one as Christ would and does.