At the moment I only have 3 - 2 boys and a girl (girl in the middle) - I would consider a 4th, but that would be my absolute limit and it's more than likely that we will stop at 3. The things that limit me are mode of delivery (have had 3 sections and would require another) and the size of our house (2 bedrooms, with plans to extend then move, but time frame is a good few years and we're already a bit cramped). Financially, we would be ok but it would make a difference to our lifestyle - I had no holidays as a child and resent it as I feel my parents prioritised having a big house over pretty much everything else, so it is definitely something that plays on my mind. Also, there are other self imposed, selfish limits like my age - I am only 30, but having given over the latter half of my 20's to having dc I am keen to have more freedom in my 30's - and going back to work at some point too! Wanting my body back is a factor too - I am now in my 6th year of non-stop pregnancy and/or breastfeeding (was still bf previous child into each subsequent pregnant, and ds2 is still bf at 17 months) and can't remember how it feels to not have someone else dependent on my body!
Having said all of the above, I still haven't totally ruled out a 4th - I think about it a lot still and dh reluctant to go for a vasectomy at the moment, despite being keen to do it in the long term, so I guess he is still not 100% sure either. I really love my family the way it is and fear messing around with it - I worry about ds2 if we were to have another boy, we would have to have a much bigger age gap than the gaps we have at present (13 months and 22 months) and I would worry about hypothetical dc4 being left out all the time because of this. I have given it a great deal of thought, but still find it hard to be completely definite - I suppose it is the same for everyone when they go beyond the average 2 dc.