Today I would like to share my latest Grand Unified Theory of Lost... An attempt to explain everything. Let's begin.
First, please note that when Jack and Hurley went to the Lighthouse, the names of the Candidates were written by numbers corresponding to degrees on a compass. There are 360 degrees on a compass, so therefore there are a total of 360 Candidates. I believe that the lighthouse opened a window to the lives of each of these possible Candidates... however, they did not open all at once. Each compass direction also refers to a specific year. One year for each Candidate. One Candidate for each year that Jacob and the Man in Black were exiled to the Island.
Since current events are taking place around the year 2007, the Man in Black and Jacob have been in exile on the Island since 1647. Each year, Jacob would ascend the stairs of the Lighthouse and turn the compass to the number of the current year of exile. A window would be opened up in space and time which would allow Jacob to travel to that place and time, where he would experience events that would reveal to him who that year's candidate is. He would then touch them in order to ensure that their destiny would eventually lead them back to the Island.
Jacob himself is a purified soul. He acheived a state of purity through purging his own evil from himself. This evil takes the form of black smoke. The Black Smoke and the Man in Black are Jacob's evil side. The price for Jacob's purity was his own imprisonment on the Island. Someone had to stay behind to prevent the Man in Black from ever leaving.
However, Jacob had the desire to spread his purity to the rest of the World. He enlisted the help of the Candidates to try and find another way of imprisoning his evil side. Both the Others AND Dharma were followers of Jacob. Jacob enlisted the help of Alvar Hanso, the Degroots, et, al. to try and find some way to "bottle" smokey. That was what the original intention behind the sonic fence was. It wasn't to keep Smokey out, it was to trap him inside.
For some reason, Jacob was unable to share this information with his original followers. That's why Richard doesn't know about the Candidates. I think Smokey was in the habit of posing as an imposter of his former self: Jacob. It was Smokey who ordered the purge, with Richard believing he was following Jacob's orders.
John Locke was, in fact, a perfect Candidate to take over for Jacob. This also made John the perfect host for Smokey. Neither Jacob nor the Man in Black could be killed until a perfect Candidate was on the Island to take over. This was the loophole. Smokey, in the form of John Locke, became Jacob's replacement. But he could not kill Jacob personally without disqualifying himself. In fact, he is now bound by a code of honor that prevents him from killing without provocation. He has to wait for other's to attack him before he attacks them back. Meanwhile, he has to find a way of disqualifying the remaining candidates, because if any of them acheive a state of peace that would allow them to replace the Man in Black (who has replaced Jacob) then MIB will become vulnerable and can be killed.
Bottom line: MIB is, in fact, Jacob. He's just the evil half of Jacob. He was able to survive when Ben killed the Good Jacob because he was in the form of John Locke, the rightful replacement for Jacob. Being in the form of Locke was also why Jacob had become mortal: He had been sucessfully replaced.