We warlocks were sent to keep an eye on you "guardians", to make sure that every time you guys found something you weren't supposed to find we would wipe your memory. Of course with space magic and the help of the traveler.