API v3 [WIP] (#6144)

* Fixed more tests * Added tags into user service * Added api-v3 auth urls * v3: fix package.json * v3: fix package.json * Fixed auth tests. Updated Authctrl response * v3: remove newrelic config file in favour of env variables * v3: upgrade some deps * switch from Q to Bluebird * v3 fix tests with deferred * Removed extra consoles.log. Changed data.data to res.data * v3 fix tests and use coroutines instead of regenerator * v3: fix tests * v3: do not await a non promise * v3: q -> bluebird * Changed id param for registration response * Updated party query and create * Ensured login callback happens after user sync * Add challenges to groups. Fixed isMemberOfGuild check * Updated party and group tests * Fixed cron test * return user.id and send analytics event before changing page * fix trailing spaces * disable redirects * Api v3 party tavern fixes (#7191) * Added check if user is in party before query * Cached party query. Prevented party request when user is not in party. Updated Party create with no invites * Update tavern ctrl to use new promise * v3: misc fixes * Api v3 task fixes (#7193) * Update task view to use _id * Added try catch to user service ops calls * v3 client: saving after syncing is complete * Fixed test broken by part sync change (#7195) * v3: fix todo scoring and try to fix production testing problem * revert changes to mongoose config * mongoose: increase keepAlive * test mongoose fix * fix: Only apply captureStackTrace if it exists on the error object * v3: fix reminders with no startDate * mongoose: use options * chore(): rename website/src -> website/server and website/public -> website/client (#7199) * v3 fix GET /groups: return an error only if an invalid type is supplied not when there are 0 results (#7203) * [API v3] Fix calls to user.ops and deleting tags (#7204) * v3: fixes calls to user.ops from views and deleting tags * v3: fix tests that use user._statsComputed * Api v3 fixes continued (#7205) * Added timzeone offset back * Added APIToken back to settings page * Fixed fetch recent messages for party * Fixed returning group description * Fixed check if user is member of challenge * Fixed party members appearing in header * Updated get myGroups param to include public groups. Fixed isMemberOf group * Fixed hourglass purchase * Fixed challenge addding tasks on first creating * Updated tests to accomidate new changes * fix: Correct checklist on client Closes #7207 * fix: Pin eslint to 2.9 * minor improvements to cron code for clarity; fix inaccurate comments; add TODOs for rest-in-inn actions * fix: Add missing type param to equip call closes #7212 * rename and reword pubChalsMinPrize to reflect that it's only for Tavern challenges * allows players to send gems to each other; other minor related changes - fixes https://github.com/HabitRPG/habitrpg/issues/7227 * fix tests for /members/transfer-gems * fix: Set gems sent notification as translatable string * chore: Remove unusued variable * fix: Remove requirement on message paramter in transfer-gems * add a missing variable declaration * chore: clarify comments on cron code * fix: Correct client request from habitrpg -> tavern * update apidoc URL in package.json Closes #7222 * Fixed start party by invites * Updated spell casting to v3 * Fixed adding and removing tags on tasks * Fixed page reload on settings change * Fixed battle monsters with friends button * Loaded completed todos when done is clicked * chore: Reinstate floating version number for eslint babel-eslint regression fixed * Fixed reload tests * change "an user" to "a user" in comments and text (no code changes) (#7257) * fix: Alert user that drops were recieved * remove userServices.js from karma.conf - it's been moved to website/client/js/services * feat: Create debug update user route * fix: Correct set cron debug function * feat: Add make admin button to debug menu * lint: Add missing semicolons in test * fix: Temporarilly comment out udpate user debug route * v3: fix _tmp for crit and streakBonus * v3: execute all actions when leaving a solo party * v3 client: fix group not found when leaving party * v3 migration: fix challenge prize * v3 cron: only save modified tasks * v3: add CHALLENGE_TASK_NOT_FOUND to valid broken reasons * v3: fix tasks chart * v3 client: fix ability to leave challenge * v3 client: fix filtering by tag and correctly show tag tooltip * v3 common: fix tags tests * v3 client: support unlinking not found challenges tasks * v3: disable Bluebird warning for missing return, fixes #7269 * feat: Separate out update-user into set-cron and make-admin debug routes * chore: Disable make admin debug route for v3 prod testing * v3: misc fixes * v3: misc fixes * v3: fix adding multiple tasks * Fixed join/leave button updates * Queried only user groups to be available when creating challenges * Fixed bulk add tasks to challenge * Synced challenge tasks after leave and join. * Fixed default selected group * Fixed challenge member info. Fixed challenge winner selection * Fixed deleting challenge tasks * Fixed particiapting filter * v3 client: fix casting spells * v3: do not log sensitive data * v3: always save user when casting spell * v3: always save user when casting spell * v3: more fixes for spells * fix typos and missing information in apidocs - fixes https://github.com/HabitRPG/habitrpg/issues/7277 (#7282) * v3: add TODO for client side spells * feat: Add modify inventory debug menu * Fixed viewing user progress on challenge * Updated tests * fix: Fix quest progress button * fix incorrect Armoire test; remove unneeded param details from apidocs; disambiguate health potion * v3: fix stealth casting * v3: fix tasks saving and selection for rebirth reroll and reset (server-only) * v3: fix auto allocation * v3 client: misc fixes * rename buyPotion and buy-potion to buyHealthPotion and buy-health-potion; fix apidoc param error * Added delete for saved challenge task * Fixed member modal on front page * adjust text in apidocs for errors / clarity / consistency / standard terminology (no code changes) (#7298) * fix bug in Rebirth test, add new tests, adjust apidocs (#7293) * Updated task model to allow setting streak (#7306) * fix: Correct missing * in apidoc comments * Api v3 challenge fixes (#7287) * Fixed join/leave button updates * Queried only user groups to be available when creating challenges * Fixed bulk add tasks to challenge * Synced challenge tasks after leave and join. * Fixed default selected group * Fixed challenge member info. Fixed challenge winner selection * Fixed deleting challenge tasks * Fixed particiapting filter * Fixed viewing user progress on challenge * Updated tests * Added delete for saved challenge task * v3: fix sorting * [API v3] add CRON_SAFE_MODE (#7286) * add CRON_SAFE_MODE to example config file, fix some bugs, add an unrelated low-priority TODO * create CRON_SAFE_MODE to disable parts of cron for use after extended outage - fixes https://github.com/HabitRPG/habitrpg/issues/7161 * fix a bug with CRON_SAFE_MODE, remove duplicated code, remove completed TODO comment * fix check for CRON_SAFE_MODE * v3 client: fix typo * adjust debug menu Modify Inventory: hungrier pets, fewer Special items, "Hide" buttons * completed To-Dos: return the 30 most recent instead of 30 oldest (#7318) * v3 migration: fix createdAt date * adjust locales text, key names, and files for Rebirth, Reset, and Fortify / ReRoll for consistency with existing strings (#7321) * v3: fix unlinking multiple tasks * v3 fix releasing pets * v3: fix authenticating with apiUrl * v3: fix typo * v3 fix client tests for unlinking * v3 client: do not show start quest button when quest is active * v3 client: fix ability to send cards * v3 client: fix misc challenge issues * v3: fix notifications * v3 client: more user friendly errors * v3 client: only load completed todos once * v3 client: fix tests * v3: move TAVERN_ID to common code * fix: Provide default type and text for new task creation in score route * fix: Provide default history [] for habit in score route * fix: Add _legacyId prop to tasks to support non-uuid identifiers * chore: Change v3 migration to use _legacyId instead of legacyId * fix: check for _legacyId in tasks if id does not exist * refactor: Extract out finding task by id or _legacyId into a function * Api v3 party quest fixes (#7341) * Fix display of add challenge message when group challenges are empty * Fixed forced quest start to update quest without reload * Fixed needing to reload when accepting party invite * Fix group leave and join reload * Fixed leave current party and join another * Updated party tests * v3 client: remove console.log statement * v3: misc fixes * v3 client: fix predicatbale random * v3: info about API v3 * v3: update footer with links to developer resources * v3: support party invitation from email * v3 client: fix chat flagging * fix: Correct get tasks route to properly get todos (#7349) * move locales strings from api-v3.json to other locales files (#7347) * move locales strings from api-v3.json: authentication strings -> front.json * move locales strings from api-v3.json: authentication strings -> tasks.json * move locales strings from api-v3.json: authentication strings -> groups.json * move locales strings from api-v3.json: authentication strings -> challenge.json * move locales strings from api-v3.json: authentication strings -> groups.json (again) * move locales strings from api-v3.json: authentication strings -> quests.json * move locales strings from api-v3.json: authentication strings -> subscriber.json * move locales strings from api-v3.json: authentication strings -> spells.json * move locales strings from api-v3.json: authentication strings -> character.json * move locales strings from api-v3.json: authentication strings -> groups.json (PMs) * move locales strings from api-v3.json: authentication strings -> npc.json * move locales strings from api-v3.json: authentication strings -> pets.json * move locales strings from api-v3.json: authentication strings -> miscellaneous * move locales strings from api-v3.json: authentication strings -> contrib.json and settings.json * move locales strings from api-v3.json: delete unused string (invalidTasksOwner), delete api-v3.json, whitespace cleanup * v3 client: fix sticky header * v3: remove unused code * v3 client: correctly redirect after inviting * Removed v2 calls from views (#7351) * v3: fix tests for challenge export * v3: fallbackto authWithHeaders if wuthWithSession or authWithUrl fails * Added force cache update when fetching new messages (#7360) * v3: fetch whole user when booting from group tto avoid issues with pre save hook expecting all data * v3: misc fixes for payments * v3: limit fields of challenge tasks that can be updated * fix(tests): never connect to NODE_DB_URI for tests * Added new route for setting last cron and updated front end * v3: fix iap url * v3: fix build and ios IAP * Changed route to user set custom day start * v3: iap accessible under /api/v3, fixes to spells and groups invitations * v3: correctly use v3 routes in client * remove XP, GP when unticking a Daily with a completed checklist - fixes https://github.com/HabitRPG/habitrpg/issues/7246 * use natural language for error message about skills on challenge tasks (#7336), fix other gramatical error * Updated ui when user rejects a guild invite (#7368) * feat: complete custom day start route Closes #7363 * fix: Correct spelling of healAll skill fix: Correct sprite name of healAll skill * fix: Change all instances of spookDust -> spookySparkles * add dateCreated to all tasks; add empty challenge object to tasks that don't have one (#7386) * add plumilla to artists for Tangle Tree in Bailey message * Fixed quest drop modal (#7377) * Fixed quest drop modal * Fixed broken party test * [API v3] Maintenance Mode (#7367) * WIP(maintenance): maintenance * WIP(maintenance): working locale features * fix(maintenance): don't translate info page target * WIP(maintenance): start adding info page * fix(maintenance): linting * feat: Add container to maintenance info page * fix(maintenance): add config.json edits Also DRY variables for main vs info pages * fix(maintenance): linting * refactor(maintenance): further slim down variables * refactor: Remove unnecessary variables * fix: Correct string interpolation in maintenace view * feat: Dynamically add time to maintenance pages * maintenance mode: do not connect to mongodb * fix(maintenance): clean up timezones etc. * fix(maintenance): remove unneeded sprite * Tavern party challenges invites fix (#7394) * Added challenges and invitations to party * Loaded tavern challenges * Updated group and quest services tests * v3: implement automatic syncing if user is not up to date * Removed unnecessary fields when updating groups and challenges (#7395) * v3: do not saved populated user * v3: correctly return user subset * Chained party promises together (#7396) * v3: $w -> splitWhitespace * use bluebird * use babel polyfill * migration: fix items * update links for v3 * Updated shortname validation to support multiple browsers * Docs changes (#7401) * chore: Clarify transfer-gems documentation * chore: Clarify api status route documentation * chore: Mark webhooks as BETA * Added tags update route. Added sort to user service (#7381) * Added tags update route. Added sort to user service * Change update tasks route to reorder tasks * Fixed linting issue * Changed params for reorder tags route * Fixed not found tag and added test * Added password confirmation when deleteing account (#7402) * fix production logging * feat(commit): push * empty commit * feat(maintenance): post-downtime news & awards (#7406) * fix exporting avatar * second attempt at fixing exporting avatar * fix production logging * s3: convert moment to date instance * fix avatar sharing and caching (30 minutes) * fix: Correct missing parameter Closes #7433 * fix: Validate challenge shortname on server * adjust text strings - fixes https://github.com/HabitRPG/habitrpg/issues/5631 and also Short Name -> Tag Name
2025-12-17 22:57:21 +01:00 · 2016-05-23 13:58:31 +02:00
parent ef3a2fc286
commit 28f2e9c356
993 changed files with 44888 additions and 12883 deletions
--- a/migrations/api_v3/challenges.js
+++ b/migrations/api_v3/challenges.js
@@ -0,0 +1,212 @@
+// Migrate challenges collection to new schema (except for members)
+
+// The console-stamp module must be installed (not included in package.json)
+
+// It requires two environment variables: MONGODB_OLD and MONGODB_NEW
+
+// Due to some big user profiles it needs more RAM than is allowed by default by v8 (arounf 1.7GB).
+// Run the script with --max-old-space-size=4096 to allow up to 4GB of RAM
+console.log('Starting migrations/api_v3/challenges.js.');
+
+require('babel-register');
+require('babel-polyfill');
+
+var Bluebird = require('bluebird');
+var MongoDB = require('mongodb');
+var nconf = require('nconf');
+var mongoose = require('mongoose');
+var _ = require('lodash');
+var uuid = require('uuid');
+var consoleStamp = require('console-stamp');
+var fs = require('fs');
+
+// Add timestamps to console messages
+consoleStamp(console);
+
+// Initialize configuration
+require('../../website/server/libs/api-v3/setupNconf')();
+
+var MONGODB_OLD = nconf.get('MONGODB_OLD');
+var MONGODB_NEW = nconf.get('MONGODB_NEW');
+
+var MongoClient = MongoDB.MongoClient;
+
+mongoose.Promise = Bluebird; // otherwise mongoose models won't work
+
+// Load new models
+var NewChallenge = require('../../website/server/models/challenge').model;
+var Tasks = require('../../website/server/models/task');
+
+// To be defined later when MongoClient connects
+var mongoDbOldInstance;
+var oldChallengeCollection;
+
+var mongoDbNewInstance;
+var newChallengeCollection;
+var newTaskCollection;
+
+var BATCH_SIZE = 1000;
+
+var processedChallenges = 0;
+var totoalProcessedTasks = 0;
+
+var newTasksIds = {}; // a map of old id -> [new id, challengeId]
+
+// Only process challenges that fall in a interval ie -> up to 0000-4000-0000-0000
+var AFTER_CHALLENGE_ID = nconf.get('AFTER_CHALLENGE_ID');
+var BEFORE_CHALLENGE_ID = nconf.get('BEFORE_CHALLENGE_ID');
+
+function processChallenges (afterId) {
+  var processedTasks = 0;
+  var lastChallenge = null;
+  var oldChallenges;
+
+  var query = {};
+
+  if (BEFORE_CHALLENGE_ID) {
+    query._id = {$lte: BEFORE_CHALLENGE_ID};
+  }
+
+  if ((afterId || AFTER_CHALLENGE_ID) && !query._id) {
+    query._id = {};
+  }
+
+  if (afterId) {
+    query._id.$gt = afterId;
+  } else if (AFTER_CHALLENGE_ID) {
+    query._id.$gt = AFTER_CHALLENGE_ID;
+  }
+
+  var batchInsertTasks = newTaskCollection.initializeUnorderedBulkOp();
+  var batchInsertChallenges = newChallengeCollection.initializeUnorderedBulkOp();
+
+  console.log(`Executing challenges query.\nMatching challenges after ${afterId ? afterId : AFTER_CHALLENGE_ID} and before ${BEFORE_CHALLENGE_ID} (included).`);
+
+  return oldChallengeCollection
+  .find(query)
+  .sort({_id: 1})
+  .limit(BATCH_SIZE)
+  .toArray()
+  .then(function (oldChallengesR) {
+    oldChallenges = oldChallengesR;
+
+    console.log(`Processing ${oldChallenges.length} challenges. Already processed ${processedChallenges} challenges and ${totoalProcessedTasks} tasks.`);
+
+    if (oldChallenges.length === BATCH_SIZE) {
+      lastChallenge = oldChallenges[oldChallenges.length - 1]._id;
+    }
+
+    oldChallenges.forEach(function (oldChallenge) {
+      var oldTasks = oldChallenge.habits.concat(oldChallenge.dailys).concat(oldChallenge.rewards).concat(oldChallenge.todos);
+      delete oldChallenge.habits;
+      delete oldChallenge.dailys;
+      delete oldChallenge.rewards;
+      delete oldChallenge.todos;
+
+      var createdAt = oldChallenge.timestamp;
+
+      oldChallenge.memberCount = oldChallenge.members.length;
+      if (oldChallenge.prize <= 0) oldChallenge.prize = 0;
+      if (!oldChallenge.name) oldChallenge.name = 'challenge name';
+      if (!oldChallenge.shortName) oldChallenge.name = 'challenge-name';
+
+      if (!oldChallenge.group) throw new Error('challenge.group is required');
+      if (!oldChallenge.leader) throw new Error('challenge.leader is required');
+
+
+      if (oldChallenge.leader === '9') {
+        oldChallenge.leader = '00000000-0000-4000-9000-000000000000';
+      }
+
+      if (oldChallenge.group === 'habitrpg') {
+        oldChallenge.group = '00000000-0000-4000-A000-000000000000';
+      }
+
+      delete oldChallenge.id;
+
+      var newChallenge = new NewChallenge(oldChallenge);
+
+      newChallenge.createdAt = createdAt;
+
+      oldTasks.forEach(function (oldTask) {
+        oldTask._id = uuid.v4();
+        oldTask._legacyId = oldTask.id; // store the old task id
+        delete oldTask.id;
+
+        oldTask.challenge = oldTask.challenge || {};
+        oldTask.challenge.id = newChallenge._id;
+
+        if (newTasksIds[oldTask._legacyId + '-' + newChallenge._id]) {
+          throw new Error('duplicate :(');
+        } else {
+          newTasksIds[oldTask._legacyId + '-' + newChallenge._id] = oldTask._id;
+        }
+
+        oldTask.tags = _.map(oldTask.tags || {}, function (tagPresent, tagId) {
+          return tagPresent && tagId;
+        }).filter(function (tag) {
+          return tag !== false;
+        });
+
+        if (!oldTask.text) oldTask.text = 'task text'; // required
+
+        oldTask.createdAt = oldTask.dateCreated;
+
+        newChallenge.tasksOrder[`${oldTask.type}s`].push(oldTask._id);
+        if (oldTask.completed) oldTask.completed = false;
+
+        var newTask = new Tasks[oldTask.type](oldTask);
+
+        batchInsertTasks.insert(newTask.toObject());
+        processedTasks++;
+      });
+
+      batchInsertChallenges.insert(newChallenge.toObject());
+    });
+
+    console.log(`Saving ${oldChallenges.length} challenges and ${processedTasks} tasks.`);
+
+    return Bluebird.all([
+      batchInsertChallenges.execute(),
+      batchInsertTasks.execute(),
+    ]);
+  })
+  .then(function () {
+    totoalProcessedTasks += processedTasks;
+    processedChallenges += oldChallenges.length;
+
+    console.log(`Saved ${oldChallenges.length} challenges and their tasks.`);
+
+    if (lastChallenge) {
+      return processChallenges(lastChallenge);
+    } else {
+      console.log('Writing newTasksIds.json...')
+      fs.writeFileSync('newTasksIds.json', JSON.stringify(newTasksIds, null, 4), 'utf8');
+      return console.log('Done!');
+    }
+  });
+}
+
+// Connect to the databases
+Bluebird.all([
+  MongoClient.connect(MONGODB_OLD),
+  MongoClient.connect(MONGODB_NEW),
+])
+.then(function (result) {
+  var oldInstance = result[0];
+  var newInstance = result[1];
+
+  mongoDbOldInstance = oldInstance;
+  oldChallengeCollection = mongoDbOldInstance.collection('challenges');
+
+  mongoDbNewInstance = newInstance;
+  newChallengeCollection = mongoDbNewInstance.collection('challenges');
+  newTaskCollection = mongoDbNewInstance.collection('tasks');
+
+  console.log(`Connected with MongoClient to ${MONGODB_OLD} and ${MONGODB_NEW}.`);
+
+  return processChallenges();
+})
+.catch(function (err) {
+  console.error(err.stack || err);
+});
--- a/migrations/api_v3/challengesMembers.js
+++ b/migrations/api_v3/challengesMembers.js
@@ -0,0 +1,143 @@
+// Migrate challenges members
+// Run AFTER users migration
+
+// The console-stamp module must be installed (not included in package.json)
+
+// It requires two environment variables: MONGODB_OLD and MONGODB_NEW
+
+// Due to some big user profiles it needs more RAM than is allowed by default by v8 (arounf 1.7GB).
+// Run the script with --max-old-space-size=4096 to allow up to 4GB of RAM
+console.log('Starting migrations/api_v3/challengesMembers.js.');
+
+require('babel-register');
+require('babel-polyfill');
+
+var Bluebird = require('bluebird');
+var MongoDB = require('mongodb');
+var nconf = require('nconf');
+var mongoose = require('mongoose');
+var _ = require('lodash');
+var uuid = require('uuid');
+var consoleStamp = require('console-stamp');
+
+// Add timestamps to console messages
+consoleStamp(console);
+
+// Initialize configuration
+require('../../website/server/libs/api-v3/setupNconf')();
+
+var MONGODB_OLD = nconf.get('MONGODB_OLD');
+var MONGODB_NEW = nconf.get('MONGODB_NEW');
+
+var MongoClient = MongoDB.MongoClient;
+
+mongoose.Promise = Bluebird; // otherwise mongoose models won't work
+
+// To be defined later when MongoClient connects
+var mongoDbOldInstance;
+var oldChallengeCollection;
+
+var mongoDbNewInstance;
+var newUserCollection;
+
+var BATCH_SIZE = 1000;
+
+var processedChallenges = 0;
+
+// Only process challenges that fall in a interval ie -> up to 0000-4000-0000-0000
+var AFTER_CHALLENGE_ID = nconf.get('AFTER_CHALLENGE_ID');
+var BEFORE_CHALLENGE_ID = nconf.get('BEFORE_CHALLENGE_ID');
+
+function processChallenges (afterId) {
+  var processedTasks = 0;
+  var lastChallenge = null;
+  var oldChallenges;
+
+  var query = {};
+
+  if (BEFORE_CHALLENGE_ID) {
+    query._id = {$lte: BEFORE_CHALLENGE_ID};
+  }
+
+  if ((afterId || AFTER_CHALLENGE_ID) && !query._id) {
+    query._id = {};
+  }
+
+  if (afterId) {
+    query._id.$gt = afterId;
+  } else if (AFTER_CHALLENGE_ID) {
+    query._id.$gt = AFTER_CHALLENGE_ID;
+  }
+
+  console.log(`Executing challenges query.\nMatching challenges after ${afterId ? afterId : AFTER_CHALLENGE_ID} and before ${BEFORE_CHALLENGE_ID} (included).`);
+
+  return oldChallengeCollection
+  .find(query)
+  .sort({_id: 1})
+  .limit(BATCH_SIZE)
+  .toArray()
+  .then(function (oldChallengesR) {
+    oldChallenges = oldChallengesR;
+
+    var promises = [];
+
+    console.log(`Processing ${oldChallenges.length} challenges. Already processed ${processedChallenges} challenges.`);
+
+    if (oldChallenges.length === BATCH_SIZE) {
+      lastChallenge = oldChallenges[oldChallenges.length - 1]._id;
+    }
+
+    oldChallenges.forEach(function (oldChallenge) {
+      // Tyler Renelle
+      oldChallenge.members.forEach(function (id, index) {
+        if (id === '9') {
+          oldChallenge.members[index] = '00000000-0000-4000-9000-000000000000';
+        }
+      });
+
+      promises.push(newUserCollection.updateMany({
+        _id: {$in: oldChallenge.members || []},
+      }, {
+        $push: {challenges: oldChallenge._id},
+      }, {multi: true}));
+    });
+
+    console.log(`Migrating members of ${oldChallenges.length} challenges.`);
+
+    return Bluebird.all(promises);
+  })
+  .then(function () {
+    processedChallenges += oldChallenges.length;
+
+    console.log(`Migrated members of ${oldChallenges.length} challenges.`);
+
+    if (lastChallenge) {
+      return processChallenges(lastChallenge);
+    } else {
+      return console.log('Done!');
+    }
+  });
+}
+
+// Connect to the databases
+Bluebird.all([
+  MongoClient.connect(MONGODB_OLD),
+  MongoClient.connect(MONGODB_NEW),
+])
+.then(function (result) {
+  var oldInstance = result[0];
+  var newInstance = result[1];
+
+  mongoDbOldInstance = oldInstance;
+  oldChallengeCollection = mongoDbOldInstance.collection('challenges');
+
+  mongoDbNewInstance = newInstance;
+  newUserCollection = mongoDbNewInstance.collection('users');
+
+  console.log(`Connected with MongoClient to ${MONGODB_OLD} and ${MONGODB_NEW}.`);
+
+  return processChallenges();
+})
+.catch(function (err) {
+  console.error(err.stack || err);
+});
--- a/migrations/api_v3/coupons.js
+++ b/migrations/api_v3/coupons.js
@@ -0,0 +1,136 @@
+// Migrate coupons collection to new schema
+
+// The console-stamp module must be installed (not included in package.json)
+
+// It requires two environment variables: MONGODB_OLD and MONGODB_NEW
+
+// Due to some big user profiles it needs more RAM than is allowed by default by v8 (arounf 1.7GB).
+// Run the script with --max-old-space-size=4096 to allow up to 4GB of RAM
+console.log('Starting migrations/api_v3/coupons.js.');
+
+require('babel-register');
+require('babel-polyfill');
+
+var Bluebird = require('bluebird');
+var MongoDB = require('mongodb');
+var nconf = require('nconf');
+var mongoose = require('mongoose');
+var _ = require('lodash');
+var uuid = require('uuid');
+var consoleStamp = require('console-stamp');
+
+// Add timestamps to console messages
+consoleStamp(console);
+
+// Initialize configuration
+require('../../website/server/libs/api-v3/setupNconf')();
+
+var MONGODB_OLD = nconf.get('MONGODB_OLD');
+var MONGODB_NEW = nconf.get('MONGODB_NEW');
+
+var MongoClient = MongoDB.MongoClient;
+
+mongoose.Promise = Bluebird; // otherwise mongoose models won't work
+
+// Load new models
+var Coupon = require('../../website/server/models/coupon').model;
+
+// To be defined later when MongoClient connects
+var mongoDbOldInstance;
+var oldCouponCollection;
+
+var mongoDbNewInstance;
+var newCouponCollection;
+
+var BATCH_SIZE = 1000;
+
+var processedCoupons = 0;
+
+// Only process coupons that fall in a interval ie -> up to 0000-4000-0000-0000
+var AFTER_COUPON_ID = nconf.get('AFTER_COUPON_ID');
+var BEFORE_COUPON_ID = nconf.get('BEFORE_COUPON_ID');
+
+function processCoupons (afterId) {
+  var processedTasks = 0;
+  var lastCoupon = null;
+  var oldCoupons;
+
+  var query = {};
+
+  if (BEFORE_COUPON_ID) {
+    query._id = {$lte: BEFORE_COUPON_ID};
+  }
+
+  if ((afterId || AFTER_COUPON_ID) && !query._id) {
+    query._id = {};
+  }
+
+  if (afterId) {
+    query._id.$gt = afterId;
+  } else if (AFTER_COUPON_ID) {
+    query._id.$gt = AFTER_COUPON_ID;
+  }
+
+  var batchInsertCoupons = newCouponCollection.initializeUnorderedBulkOp();
+
+  console.log(`Executing coupons query.\nMatching coupons after ${afterId ? afterId : AFTER_COUPON_ID} and before ${BEFORE_COUPON_ID} (included).`);
+
+  return oldCouponCollection
+  .find(query)
+  .sort({_id: 1})
+  .limit(BATCH_SIZE)
+  .toArray()
+  .then(function (oldCouponsR) {
+    oldCoupons = oldCouponsR;
+
+    console.log(`Processing ${oldCoupons.length} coupons. Already processed ${processedCoupons} coupons.`);
+
+    if (oldCoupons.length === BATCH_SIZE) {
+      lastCoupon = oldCoupons[oldCoupons.length - 1]._id;
+    }
+
+    oldCoupons.forEach(function (oldCoupon) {
+      var newCoupon = new Coupon(oldCoupon);
+
+      batchInsertCoupons.insert(newCoupon.toObject());
+    });
+
+    console.log(`Saving ${oldCoupons.length} coupons.`);
+
+    return batchInsertCoupons.execute();
+  })
+  .then(function () {
+    processedCoupons += oldCoupons.length;
+
+    console.log(`Saved ${oldCoupons.length} coupons.`);
+
+    if (lastCoupon) {
+      return processCoupons(lastCoupon);
+    } else {
+      return console.log('Done!');
+    }
+  });
+}
+
+// Connect to the databases
+Bluebird.all([
+  MongoClient.connect(MONGODB_OLD),
+  MongoClient.connect(MONGODB_NEW),
+])
+.then(function (result) {
+  var oldInstance = result[0];
+  var newInstance = result[1];
+
+  mongoDbOldInstance = oldInstance;
+  oldCouponCollection = mongoDbOldInstance.collection('coupons');
+
+  mongoDbNewInstance = newInstance;
+  newCouponCollection = mongoDbNewInstance.collection('coupons');
+
+  console.log(`Connected with MongoClient to ${MONGODB_OLD} and ${MONGODB_NEW}.`);
+
+  return processCoupons();
+})
+.catch(function (err) {
+  console.error(err.stack || err);
+});
--- a/migrations/api_v3/emailUnsubscriptions.js
+++ b/migrations/api_v3/emailUnsubscriptions.js
@@ -0,0 +1,137 @@
+// Migrate unsubscriptions collection to new schema
+
+// The console-stamp module must be installed (not included in package.json)
+
+// It requires two environment variables: MONGODB_OLD and MONGODB_NEW
+
+// Due to some big user profiles it needs more RAM than is allowed by default by v8 (arounf 1.7GB).
+// Run the script with --max-old-space-size=4096 to allow up to 4GB of RAM
+console.log('Starting migrations/api_v3/unsubscriptions.js.');
+
+require('babel-register');
+require('babel-polyfill');
+
+var Bluebird = require('bluebird');
+var MongoDB = require('mongodb');
+var nconf = require('nconf');
+var mongoose = require('mongoose');
+var _ = require('lodash');
+var uuid = require('uuid');
+var consoleStamp = require('console-stamp');
+
+// Add timestamps to console messages
+consoleStamp(console);
+
+// Initialize configuration
+require('../../website/server/libs/api-v3/setupNconf')();
+
+var MONGODB_OLD = nconf.get('MONGODB_OLD');
+var MONGODB_NEW = nconf.get('MONGODB_NEW');
+
+var MongoClient = MongoDB.MongoClient;
+
+mongoose.Promise = Bluebird; // otherwise mongoose models won't work
+
+// Load new models
+var EmailUnsubscription = require('../../website/server/models/emailUnsubscription').model;
+
+// To be defined later when MongoClient connects
+var mongoDbOldInstance;
+var oldUnsubscriptionCollection;
+
+var mongoDbNewInstance;
+var newUnsubscriptionCollection;
+
+var BATCH_SIZE = 1000;
+
+var processedUnsubscriptions = 0;
+
+// Only process unsubscriptions that fall in a interval ie -> up to 0000-4000-0000-0000
+var AFTER_UNSUBSCRIPTION_ID = nconf.get('AFTER_UNSUBSCRIPTION_ID');
+var BEFORE_UNSUBSCRIPTION_ID = nconf.get('BEFORE_UNSUBSCRIPTION_ID');
+
+function processUnsubscriptions (afterId) {
+  var processedTasks = 0;
+  var lastUnsubscription = null;
+  var oldUnsubscriptions;
+
+  var query = {};
+
+  if (BEFORE_UNSUBSCRIPTION_ID) {
+    query._id = {$lte: BEFORE_UNSUBSCRIPTION_ID};
+  }
+
+  if ((afterId || AFTER_UNSUBSCRIPTION_ID) && !query._id) {
+    query._id = {};
+  }
+
+  if (afterId) {
+    query._id.$gt = afterId;
+  } else if (AFTER_UNSUBSCRIPTION_ID) {
+    query._id.$gt = AFTER_UNSUBSCRIPTION_ID;
+  }
+
+  var batchInsertUnsubscriptions = newUnsubscriptionCollection.initializeUnorderedBulkOp();
+
+  console.log(`Executing unsubscriptions query.\nMatching unsubscriptions after ${afterId ? afterId : AFTER_UNSUBSCRIPTION_ID} and before ${BEFORE_UNSUBSCRIPTION_ID} (included).`);
+
+  return oldUnsubscriptionCollection
+  .find(query)
+  .sort({_id: 1})
+  .limit(BATCH_SIZE)
+  .toArray()
+  .then(function (oldUnsubscriptionsR) {
+    oldUnsubscriptions = oldUnsubscriptionsR;
+
+    console.log(`Processing ${oldUnsubscriptions.length} unsubscriptions. Already processed ${processedUnsubscriptions} unsubscriptions.`);
+
+    if (oldUnsubscriptions.length === BATCH_SIZE) {
+      lastUnsubscription = oldUnsubscriptions[oldUnsubscriptions.length - 1]._id;
+    }
+
+    oldUnsubscriptions.forEach(function (oldUnsubscription) {
+      oldUnsubscription.email = oldUnsubscription.email.toLowerCase();
+      var newUnsubscription = new EmailUnsubscription(oldUnsubscription);
+
+      batchInsertUnsubscriptions.insert(newUnsubscription.toObject());
+    });
+
+    console.log(`Saving ${oldUnsubscriptions.length} unsubscriptions.`);
+
+    return batchInsertUnsubscriptions.execute();
+  })
+  .then(function () {
+    processedUnsubscriptions += oldUnsubscriptions.length;
+
+    console.log(`Saved ${oldUnsubscriptions.length} unsubscriptions.`);
+
+    if (lastUnsubscription) {
+      return processUnsubscriptions(lastUnsubscription);
+    } else {
+      return console.log('Done!');
+    }
+  });
+}
+
+// Connect to the databases
+Bluebird.all([
+  MongoClient.connect(MONGODB_OLD),
+  MongoClient.connect(MONGODB_NEW),
+])
+.then(function (result) {
+  var oldInstance = result[0];
+  var newInstance = result[1];
+
+  mongoDbOldInstance = oldInstance;
+  oldUnsubscriptionCollection = mongoDbOldInstance.collection('emailunsubscriptions');
+
+  mongoDbNewInstance = newInstance;
+  newUnsubscriptionCollection = mongoDbNewInstance.collection('emailunsubscriptions');
+
+  console.log(`Connected with MongoClient to ${MONGODB_OLD} and ${MONGODB_NEW}.`);
+
+  return processUnsubscriptions();
+})
+.catch(function (err) {
+  console.error(err.stack || err);
+});
--- a/migrations/api_v3/groups.js
+++ b/migrations/api_v3/groups.js
@@ -0,0 +1,211 @@
+/*
+  members are not stored anymore
+  invites are not stored anymore
+
+  tavern id and leader must be updated
+*/
+
+// Migrate groups collection to new schema
+// Run AFTER users migration
+
+// The console-stamp module must be installed (not included in package.json)
+
+// It requires two environment variables: MONGODB_OLD and MONGODB_NEW
+
+// Due to some big user profiles it needs more RAM than is allowed by default by v8 (arounf 1.7GB).
+// Run the script with --max-old-space-size=4096 to allow up to 4GB of RAM
+console.log('Starting migrations/api_v3/groups.js.');
+
+require('babel-register');
+require('babel-polyfill');
+
+var Bluebird = require('bluebird');
+var MongoDB = require('mongodb');
+var nconf = require('nconf');
+var mongoose = require('mongoose');
+var _ = require('lodash');
+var uuid = require('uuid');
+var consoleStamp = require('console-stamp');
+
+// Add timestamps to console messages
+consoleStamp(console);
+
+// Initialize configuration
+require('../../website/server/libs/api-v3/setupNconf')();
+
+var MONGODB_OLD = nconf.get('MONGODB_OLD');
+var MONGODB_NEW = nconf.get('MONGODB_NEW');
+
+var MongoClient = MongoDB.MongoClient;
+
+mongoose.Promise = Bluebird; // otherwise mongoose models won't work
+
+// Load new models
+var NewGroup = require('../../website/server/models/group').model;
+
+var TAVERN_ID = require('../../website/server/models/group').TAVERN_ID;
+
+// To be defined later when MongoClient connects
+var mongoDbOldInstance;
+var oldGroupCollection;
+
+var mongoDbNewInstance;
+var newGroupCollection;
+var newUserCollection;
+
+var BATCH_SIZE = 1000;
+
+var processedGroups = 0;
+
+// Only process groups that fall in a interval ie -> up to 0000-4000-0000-0000
+var AFTER_GROUP_ID = nconf.get('AFTER_GROUP_ID');
+var BEFORE_GROUP_ID = nconf.get('BEFORE_GROUP_ID');
+
+function processGroups (afterId) {
+  var processedTasks = 0;
+  var lastGroup = null;
+  var oldGroups;
+
+  var query = {};
+
+  if (BEFORE_GROUP_ID) {
+    query._id = {$lte: BEFORE_GROUP_ID};
+  }
+
+  if ((afterId || AFTER_GROUP_ID) && !query._id) {
+    query._id = {};
+  }
+
+  if (afterId) {
+    query._id.$gt = afterId;
+  } else if (AFTER_GROUP_ID) {
+    query._id.$gt = AFTER_GROUP_ID;
+  }
+
+  var batchInsertGroups = newGroupCollection.initializeUnorderedBulkOp();
+
+  console.log(`Executing groups query.\nMatching groups after ${afterId ? afterId : AFTER_GROUP_ID} and before ${BEFORE_GROUP_ID} (included).`);
+
+  return oldGroupCollection
+  .find(query)
+  .sort({_id: 1})
+  .limit(BATCH_SIZE)
+  .toArray()
+  .then(function (oldGroupsR) {
+    oldGroups = oldGroupsR;
+
+    var promises = [];
+
+    console.log(`Processing ${oldGroups.length} groups. Already processed ${processedGroups} groups.`);
+
+    if (oldGroups.length === BATCH_SIZE) {
+      lastGroup = oldGroups[oldGroups.length - 1]._id;
+    }
+
+    oldGroups.forEach(function (oldGroup) {
+      if ((!oldGroup.privacy || oldGroup.privacy === 'private') && (!oldGroup.members || oldGroup.members.length === 0)) return; // delete empty private groups TODO must also delete challenges or this won't work
+
+      oldGroup.members = oldGroup.members || [];
+      oldGroup.memberCount = oldGroup.members ? oldGroup.members.length : 0;
+      oldGroup.challengeCount = oldGroup.challenges ? oldGroup.challenges.length : 0;
+
+      if (!oldGroup.balance <= 0) oldGroup.balance = 0;
+      if (!oldGroup.name) oldGroup.name = 'group name';
+      if (!oldGroup.leaderOnly) oldGroup.leaderOnly = {};
+      if (!oldGroup.leaderOnly.challenges) oldGroup.leaderOnly.challenges = false;
+
+      // Tavern
+      if (oldGroup._id === 'habitrpg') {
+        oldGroup._id = TAVERN_ID;
+        oldGroup.leader = '7bde7864-ebc5-4ee2-a4b7-1070d464cdb0'; // Siena Leslie
+      }
+
+      if (!oldGroup.type) {
+        // throw new Error('group.type is required');
+        oldGroup.type = 'guild';
+      }
+
+      if (!oldGroup.leader) {
+        if (oldGroup.members && oldGroup.members.length > 0) {
+          oldGroup.leader = oldGroup.members[0];
+        } else {
+          throw new Error('group.leader is required and no member available!');
+        }
+      }
+
+      if (!oldGroup.privacy) {
+        // throw new Error('group.privacy is required');
+        oldGroup.privacy = 'private';
+      }
+
+      var updateMembers = {};
+
+      if (oldGroup.type === 'guild') {
+        updateMembers.$push = {guilds: oldGroup._id};
+      } else if (oldGroup.type === 'party') {
+        updateMembers.$set = {'party._id': oldGroup._id};
+      }
+
+      if (oldGroup.members) {
+        // Tyler Renelle
+        oldGroup.members.forEach(function (id, index) {
+          if (id === '9') {
+            oldGroup.members[index] = '00000000-0000-4000-9000-000000000000';
+          }
+        });
+
+        promises.push(newUserCollection.updateMany({
+          _id: {$in: oldGroup.members},
+        }, updateMembers, {multi: true}));
+      }
+
+      var newGroup = new NewGroup(oldGroup);
+
+      batchInsertGroups.insert(newGroup.toObject());
+    });
+
+    console.log(`Saving ${oldGroups.length} groups and migrating members to users collection.`);
+
+    promises.push(batchInsertGroups.execute());
+    return Bluebird.all(promises);
+  })
+  .then(function () {
+    processedGroups += oldGroups.length;
+
+    console.log(`Saved ${oldGroups.length} groups and migrated their members to the user collection.`);
+
+    if (lastGroup) {
+      return processGroups(lastGroup);
+    } else {
+      return console.log('Done!');
+    }
+  });
+}
+
+// Connect to the databases
+Bluebird.all([
+  MongoClient.connect(MONGODB_OLD),
+  MongoClient.connect(MONGODB_NEW),
+])
+.then(function (result) {
+  var oldInstance = result[0];
+  var newInstance = result[1];
+
+  mongoDbOldInstance = oldInstance;
+  oldGroupCollection = mongoDbOldInstance.collection('groups');
+
+  mongoDbNewInstance = newInstance;
+  newGroupCollection = mongoDbNewInstance.collection('groups');
+  newUserCollection = mongoDbNewInstance.collection('users');
+
+  console.log(`Connected with MongoClient to ${MONGODB_OLD} and ${MONGODB_NEW}.`);
+
+  // First delete the tavern group created by having required the group model
+  return newGroupCollection.deleteOne({_id: TAVERN_ID});
+})
+.then(function () {
+  return processGroups();
+})
+.catch(function (err) {
+  console.error(err.stack || err);
+});
--- a/migrations/api_v3/indexes.js
+++ b/migrations/api_v3/indexes.js
@@ -0,0 +1,52 @@
+/*
+  DEFINE BEFORE MIGRATING
+
+  tasks: userId OK (sparse?), challenge.id OK (sparse?), challenge.taskId OK (sparse?), type? completed?
+  users:
+    id & apiToken, OK
+    auth.facebook.emails.value OK -> unique and sparse?,
+    auth.facebook.id - unique and sparse, OK
+    auth.local.email - unique and sparse, OK
+    auth.local.lowerCaseUsername, OK
+    auth.local.username - unique OK
+    auth.local.username & auth.local.hashed_password?,
+    auth.timestamps.created?, OK
+    auth.timestamps.loggedin?, OK
+    backer.tier -1 OK
+    { "contributor.admin" : 1 , "contributor.level" : -1 , "backer.npc" : -1 , "profile.name" : 1}
+    { "contributor.admin" : 1.0} NO, see ^
+    { "contributor.level" : 1.0} OK
+    { "contributor.level" : 1.0 , "purchased.plan.customerId" : 1.0} ?
+    NO { "flags.lastWeeklyRecap" : 1 , "_id" : 1 , "preferences.emailNotifications.unsubscribeFromAll" : 1 , "preferences.emailNotifications.weeklyRecaps" : 1}
+    { "invitations.guilds.id" : 1} OK
+    { "invitations.party.id" : 1} OK
+    OK { "preferences.sleep" : 1 , "_id" : 1 , "flags.lastWeeklyRecap" : 1 , "preferences.emailNotifications.unsubscribeFromAll" : 1 , "preferences.emailNotifications.weeklyRecaps" : 1}
+    OK { "preferences.sleep" : 1 , "_id" : 1 , "lastCron" : 1 , "preferences.emailNotifications.importantAnnouncements" : 1 , "preferences.emailNotifications.unsubscribeFromAll" : 1 , "flags.recaptureEmailsPhase" : 1}
+    profile.name ? OK
+    { "purchased.plan.customerId" : 1.0} OK
+    { "purchased.plan.paymentMethod" : 1.0} OK
+
+    guilds OK
+    party.id OK
+    challenges OK
+  challenges:
+    { "_id" : 1.0 , "__v" : 1.0} ? NO
+    { "_id" : 1.0 , "official" : -1.0 , "timestamp" : -1.0}
+    { "group" : 1.0 , "official" : -1.0 , "timestamp" : -1.0} OK
+    { "leader" : 1.0 , "official" : -1.0 , "timestamp" : -1.0} OK
+    { "members" : 1.0 , "official" : -1.0 , "timestamp" : -1.0} ? NO
+    { "official" : -1 , "timestamp" : -1} ?
+    { "official" : -1 , "timestamp" : -1, "_id": 1} ?
+  groups:
+    { "_id" : 1 , "quest.key" : 1} ?
+    { "_id" : 1.0 , "__v" : 1.0} ?
+    { "_id" : 1.0 , "privacy" : 1.0 , "members" : 1.0} ? NO
+    { "members" : 1.0 , "type" : 1.0 , "memberCount" : -1.0} ? NO
+    { "members" : 1} ? NO
+    { "privacy" : 1.0 , "memberCount" : -1.0} ?
+    { "privacy" : 1.0} OK
+    { "type" : 1 , "privacy" : 1} ?
+    { "type" : 1.0 , "members" : 1.0} ? NO
+    { "type" : 1} ? OK
+  emailUnsubscriptions: email unique OK
+*/
--- a/migrations/api_v3/users.js
+++ b/migrations/api_v3/users.js
@@ -0,0 +1,262 @@
+// Migrate users collection to new schema
+// This should run AFTER challenges migration
+
+// The console-stamp module must be installed (not included in package.json)
+
+// It requires two environment variables: MONGODB_OLD and MONGODB_NEW
+
+// Due to some big user profiles it needs more RAM than is allowed by default by v8 (arounf 1.7GB).
+// Run the script with --max-old-space-size=4096 to allow up to 4GB of RAM
+console.log('Starting migrations/api_v3/users.js.');
+
+require('babel-register');
+require('babel-polyfill');
+
+var Bluebird = require('bluebird');
+var MongoDB = require('mongodb');
+var nconf = require('nconf');
+var mongoose = require('mongoose');
+var _ = require('lodash');
+var uuid = require('uuid');
+var consoleStamp = require('console-stamp');
+var common = require('../../common');
+var moment = require('moment');
+
+// Add timestamps to console messages
+consoleStamp(console);
+
+// Initialize configuration
+require('../../website/server/libs/api-v3/setupNconf')();
+
+var MONGODB_OLD = nconf.get('MONGODB_OLD');
+var MONGODB_NEW = nconf.get('MONGODB_NEW');
+
+var taskDefaults = common.taskDefaults;
+var MongoClient = MongoDB.MongoClient;
+
+mongoose.Promise = Bluebird; // otherwise mongoose models won't work
+
+// Load new models
+var NewUser = require('../../website/server/models/user').model;
+var NewTasks = require('../../website/server/models/task');
+
+// To be defined later when MongoClient connects
+var mongoDbOldInstance;
+var oldUserCollection;
+
+var mongoDbNewInstance;
+var newUserCollection;
+var newTaskCollection;
+
+var BATCH_SIZE = 1000;
+
+var processedUsers = 0;
+var totoalProcessedTasks = 0;
+
+var challengeTaskWithMatchingId = 0;
+var challengeTaskNoMatchingId = 0;
+
+// Load the new tasks ids for challenges tasks
+var newTasksIds = require('./newTasksIds.json');
+
+// Only process users that fall in a interval ie up to -> 0000-4000-0000-0000
+var AFTER_USER_ID = nconf.get('AFTER_USER_ID');
+var BEFORE_USER_ID = nconf.get('BEFORE_USER_ID');
+
+function processUsers (afterId) {
+  var processedTasks = 0;
+  var lastUser = null;
+  var oldUsers;
+
+  var now = new Date();
+
+  var query = {};
+
+  if (BEFORE_USER_ID) {
+    query._id = {$lte: BEFORE_USER_ID};
+  }
+
+  if ((afterId || AFTER_USER_ID) && !query._id) {
+    query._id = {};
+  }
+
+  if (afterId) {
+    query._id.$gt = afterId;
+  } else if (AFTER_USER_ID) {
+    query._id.$gt = AFTER_USER_ID;
+  }
+
+  var batchInsertTasks = newTaskCollection.initializeUnorderedBulkOp();
+  var batchInsertUsers = newUserCollection.initializeUnorderedBulkOp();
+
+  console.log(`Executing users query.\nMatching users after ${afterId ? afterId : AFTER_USER_ID} and before ${BEFORE_USER_ID} (included).`);
+
+  return oldUserCollection
+  .find(query)
+  .sort({_id: 1})
+  .limit(BATCH_SIZE)
+  .toArray()
+  .then(function (oldUsersR) {
+    oldUsers = oldUsersR;
+
+    console.log(`Processing ${oldUsers.length} users. Already processed ${processedUsers} users and ${totoalProcessedTasks} tasks.`);
+
+    if (oldUsers.length === BATCH_SIZE) {
+      lastUser = oldUsers[oldUsers.length - 1]._id;
+    }
+
+    oldUsers.forEach(function (oldUser) {
+      var oldTasks = oldUser.habits.concat(oldUser.dailys).concat(oldUser.rewards).concat(oldUser.todos);
+      delete oldUser.habits;
+      delete oldUser.dailys;
+      delete oldUser.rewards;
+      delete oldUser.todos;
+
+      delete oldUser.id;
+
+      // spookDust -> spookySparkles
+
+      if (oldUser.achievements && oldUser.achievements.spookDust) {
+        oldUser.achievements.spookySparkles = oldUser.achievements.spookDust;
+        delete oldUser.achievements.spookDust;
+      }
+
+      if (oldUser.items && oldUser.items.special && oldUser.items.special.spookDust) {
+        oldUser.items.special.spookySparkles = oldUser.items.special.spookDust;
+        delete oldUser.items.special.spookDust;
+      }
+
+      if (oldUser.stats && oldUser.stats.buffs && oldUser.stats.buffs.spookySparkles) {
+        oldUser.stats.buffs.spookySparkles = oldUser.stats.buffs.spookDust;
+        delete oldUser.stats.buffs.spookDust;
+      }
+
+      // end spookDust -> spookySparkles
+
+      oldUser.tags = oldUser.tags.map(function (tag) {
+        return {
+          id: tag.id,
+          name: tag.name || 'tag name',
+          challenge: tag.challenge,
+        };
+      });
+
+      if (oldUser._id === '9') { // Tyler Renelle
+        oldUser._id = '00000000-0000-4000-9000-000000000000';
+      }
+
+      var newUser = new NewUser(oldUser);
+      var isSubscribed = newUser.isSubscribed();
+
+      oldTasks.forEach(function (oldTask) {
+        oldTask._id = uuid.v4(); // create a new unique uuid
+        oldTask.userId = newUser._id;
+        oldTask._legacyId = oldTask.id; // store the old task id
+        delete oldTask.id;
+
+        oldTask.challenge = oldTask.challenge || {};
+        if (oldTask.challenge.id) {
+          if (oldTask.challenge.broken) {
+            oldTask.challenge.taskId = oldTask._legacyId;
+          } else {
+            var newId = newTasksIds[oldTask._legacyId + '-' + oldTask.challenge.id];
+
+            // Challenges' tasks ids changed
+            if (!newId && !oldTask.challenge.broken) {
+              challengeTaskNoMatchingId++;
+              oldTask.challenge.taskId = oldTask._legacyId;
+              oldTask.challenge.broken = 'CHALLENGE_TASK_NOT_FOUND';
+            } else {
+              challengeTaskWithMatchingId++;
+              oldTask.challenge.taskId = newId;
+            }
+          }
+        }
+
+        // Delete old completed todos
+        if (oldTask.type === 'todo' && oldTask.completed && (!oldTask.challenge.id || oldTask.challenge.broken)) {
+          if (moment(now).subtract(isSubscribed ? 90 : 30, 'days').toDate() > moment(oldTask.dateCompleted).toDate()) {
+            return;
+          }
+        }
+
+        oldTask.createdAt = oldTask.dateCreated;
+
+        if (!oldTask.text) oldTask.text = 'task text'; // required
+        oldTask.tags = _.map(oldTask.tags, function (tagPresent, tagId) {
+          return tagPresent && tagId;
+        }).filter(function (tag) {
+          return tag !== false;
+        });
+
+        if (oldTask.type !== 'todo' || (oldTask.type === 'todo' && !oldTask.completed)) {
+          newUser.tasksOrder[`${oldTask.type}s`].push(oldTask._id);
+        }
+
+        var allTasksFields = ['_id', 'type', 'text', 'notes', 'tags', 'value', 'priority', 'attribute', 'challenge', 'reminders', 'userId', '_legacyId', 'createdAt'];
+        // using mongoose models is too slow
+        if (oldTask.type === 'habit') {
+          oldTask = _.pick(oldTask, allTasksFields.concat(['history', 'up', 'down']));
+        } else if (oldTask.type === 'daily') {
+          oldTask = _.pick(oldTask, allTasksFields.concat(['completed', 'collapseChecklist', 'checklist', 'history', 'frequency', 'everyX', 'startDate', 'repeat', 'streak']));
+        } else if (oldTask.type === 'todo') {
+          oldTask = _.pick(oldTask, allTasksFields.concat(['completed', 'collapseChecklist', 'checklist', 'date', 'dateCompleted']));
+        } else if (oldTask.type === 'reward') {
+          oldTask = _.pick(oldTask, allTasksFields);
+        } else {
+          throw new Error('Task with no or invalid type!');
+        }
+
+        batchInsertTasks.insert(taskDefaults(oldTask));
+        processedTasks++;
+      });
+
+      batchInsertUsers.insert(newUser.toObject());
+    });
+
+    console.log(`Saving ${oldUsers.length} users and ${processedTasks} tasks.`);
+
+    return Bluebird.all([
+      batchInsertUsers.execute(),
+      batchInsertTasks.execute(),
+    ]);
+  })
+  .then(function () {
+    totoalProcessedTasks += processedTasks;
+    processedUsers += oldUsers.length;
+
+    console.log(`Saved ${oldUsers.length} users and their tasks.`);
+    console.log('Challenges\' tasks no matching id: ', challengeTaskNoMatchingId);
+    console.log('Challenges\' tasks with matching id: ', challengeTaskWithMatchingId);
+
+    if (lastUser) {
+      return processUsers(lastUser);
+    } else {
+      return console.log('Done!');
+    }
+  });
+}
+
+// Connect to the databases
+Bluebird.all([
+  MongoClient.connect(MONGODB_OLD),
+  MongoClient.connect(MONGODB_NEW),
+])
+.then(function (result) {
+  var oldInstance = result[0];
+  var newInstance = result[1];
+
+  mongoDbOldInstance = oldInstance;
+  oldUserCollection = mongoDbOldInstance.collection('users');
+
+  mongoDbNewInstance = newInstance;
+  newUserCollection = mongoDbNewInstance.collection('users');
+  newTaskCollection = mongoDbNewInstance.collection('tasks');
+
+  console.log(`Connected with MongoClient to ${MONGODB_OLD} and ${MONGODB_NEW}.`);
+
+  return processUsers();
+})
+.catch(function (err) {
+  console.error(err.stack || err);
+});