{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Demo 1 (PS/PE): Project Creation\n",
    "\n",
    "*Written and last updated July 19, 2025 by Sedona Ewbank, snewbank@stanford.edu*\n",
    "\n",
    "The purpose of this demo is to demonstrate how to create and edit a MARIPoSA project via the Python module."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "5596d31f01357060"
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 1.0 Import and Setup"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "cc54455c4d7d10a9"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "import os\n",
    "import sys\n",
    "import importlib\n",
    "\n",
    "#This block only important for running as script\n",
    "#script_dir = os.path.dirname(os.path.abspath(__file__))\n",
    "mariposa_dir = \"/Users/snewbank/PycharmProjects/MARIPoSA\"\n",
    "utils_dir = os.path.join(mariposa_dir, 'utils')\n",
    "sys.path.append(utils_dir)\n",
    "sys.path.append(mariposa_dir)\n",
    "\n",
    "#import utils\n",
    "from utils import metadata, analyze, plot, simulate\n",
    "\n",
    "importlib.reload(metadata)\n",
    "importlib.reload(analyze)\n",
    "importlib.reload(plot)\n",
    "importlib.reload(simulate)\n",
    "\n",
    "demo_dir=\"/Users/snewbank/Behavior/MARIPoSA_demo_data/\""
   ],
   "metadata": {
    "collapsed": true,
    "ExecuteTime": {
     "end_time": "2025-09-17T23:40:53.625617Z",
     "start_time": "2025-09-17T23:40:51.124125Z"
    }
   },
   "id": "initial_id",
   "execution_count": 1
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 1.1: Creating a pose segmentation project\n",
    "\n",
    "Two types of projects can be created: pose segmentation (to handle data output from Keypoint-MoSeq, B-SOiD, or VAME) and pose estimation (to handle data output from DeepLabCut or OpenFace). We will start by making a pose segmentation project, using command metadata.create_PS_project:"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "5e8c2866cdcbcf6c"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function create_PS_project in module utils.metadata:\n",
      "\n",
      "create_PS_project(project_name, data_directory, data_source, output_directory, fps, n_modules)\n",
      "    Make project directory and write config_PS.yaml file.\n",
      "    \n",
      "    :param project_name: what to call your creation\n",
      "    :param data_directory: path to source data\n",
      "    :param data_source: B-SOiD, VAME, or Keypoint-MoSeq\n",
      "    :param output_directory: path where output directory and config file should be created\n",
      "    :param fps: frames per second\n",
      "    :param n_modules: number of pose states / modules\n",
      "    :return: path to saved config (in specified output directory)\n"
     ]
    }
   ],
   "source": [
    "help(metadata.create_PS_project)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-09-17T23:40:53.630705Z",
     "start_time": "2025-09-17T23:40:53.625568Z"
    }
   },
   "id": "babe084b759ef49c",
   "execution_count": 2
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2025-09-17 16:40:53.649587] Made project 250917_test and saved at /Users/snewbank/Behavior/MARIPoSA_demo_data//250917_test/config_PS.yaml\n"
     ]
    }
   ],
   "source": [
    "project_name=\"test\"\n",
    "data_directory=demo_dir+\"PS-TEST-25-05-20-rat-VAME/results/\"\n",
    "data_source=\"VAME\"\n",
    "output_directory=demo_dir\n",
    "fps=30\n",
    "n_modules=30\n",
    "\n",
    "config_path = metadata.create_PS_project(project_name,\n",
    "                                         data_directory,\n",
    "                                         data_source,\n",
    "                                         output_directory,\n",
    "                                         fps,\n",
    "                                         n_modules)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-09-17T23:40:53.664280Z",
     "start_time": "2025-09-17T23:40:53.631053Z"
    }
   },
   "id": "88b760a308306959",
   "execution_count": 3
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now we can load our config from the path it was saved at using metadata.load_config and the path to config (output by the project creation function)."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "3a2838aca8eb5ccd"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'project_name': '250917_test', 'data_directory': '/Users/snewbank/Behavior/MARIPoSA_demo_data/PS-TEST-25-05-20-rat-VAME/results/', 'project_directory': '/Users/snewbank/Behavior/MARIPoSA_demo_data//250917_test', 'data_type': 'Pose segmentation', 'data_source': 'VAME', 'n_modules': '30', 'fps': '30', 'vame_model_name': 'hmm-30', 'project_files': ['2023-05-09_15-52-24_bm01_badinjxnDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-09_16-02-49_bm02_k5DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-09_16-36-04_bm03_salDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-09_17-02-19_bm04_k1DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-09_17-41-11_bf01_nal10k10DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-09_18-06-23_bf02_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-09_19-16-35_bf03_k5DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-09_20-12-22_bf04_k25DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_11-00-30_bm01_k5DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_11-30-56_bm02_salDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_12-01-26_bm03_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_13-16-30_bm04_salDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_13-44-36_bf01_cocDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_14-09-38_bf02_k1DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_16-47-52_bf03_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_18-18-22_bf04_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-20_17-11-22_bm01_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-20_18-17-01_bm02_cocDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_13-24-45_bm03_k25DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_13-52-07_bm04_halDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_14-22-22_bm04_halketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_14-53-10_bf01_halDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_15-23-17_bf01_halketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_16-17-57_bf02_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_17-26-26_bf03_halDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_17-58-44_bf03_halketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_18-25-03_bf04_halDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_18-58-40_bf04_halketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-25_13-59-51_bm01_cocDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-25_14-30-54_bm02_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-26_12-51-23_bm03_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-26_13-18-27_bm04_k5DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-26_13-47-55_bf01_salDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-26_14-27-59_bf02_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-26_14-56-26_bf03_salDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-26_15-22-23_bf04_k1DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-30_13-39-15_bm01_k25DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-30_14-08-51_bm02_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-31_11-26-43_bm04_badinjxnDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-31_11-34-17_bm03_k1DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-31_12-02-23_bf02_cocDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-31_17-08-06_bf01_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-31_18-28-16_bf03_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-31_19-08-34_bf04_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-09_16-55-57_bm01_halDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-09_17-33-31_bm01_halketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-09_18-18-49_bm02_k25DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-09_18-53-52_bm03_halDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-09_19-25-00_bm03_halketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-09_20-06-37_bm04_nalket_cutshortDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-10_20-30-02_bf01_k1DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-10_20-57-51_bf02_k25DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-10_21-26-14_bf03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-10_21-52-24_bf04_k5DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_17-16-18_bm01_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_17-54-08_bm02_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_18-21-57_bm03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_19-15-24_bm04_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_19-46-22_bf01_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_20-51-12_bf02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_21-28-12_bf04_salivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_22-01-50_bf03_salivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-20_15-03-24_bm02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-20_15-41-01_bm03_salivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-20_16-13-48_bm04_salivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-20_16-46-29_bf01_k5DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-20_17-21-51_bf02_salivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-20_17-54-10_bf03_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-20_19-10-38_bf04_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-26_15-35-45_bf01_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-26_16-15-58_bf02_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-26_16-49-06_bf03_co10DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-26_17-16-08_bf04_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-26_18-27-06_bm01_k25DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-26_19-00-05_bm02_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-27_10-14-36_bm03_co10DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-27_10-47-10_bm04_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_15-04-47_bm01_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_16-15-21_bm02_salivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_16-46-00_bm03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_17-54-37_bm04_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_19-20-49_bf01_salivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_19-48-04_bf02_salDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_20-27-25_bf03_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_20-52-16_bf04_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-08_14-12-55_bf01_k5iv_badinjxnDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-08_14-42-24_bf02_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-08_15-13-11_bf03_k25DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-08_15-47-29_bm01_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-08_16-19-27_bm02_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-08_16-55-20_bm03_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-13_14-33-00_bf03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-13_15-45-26_bm03_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-13_16-56-46_bm04_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-14_13-56-09_bm01_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-14_15-05-22_bm02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-14_16-25-47_bf01_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-14_17-04-10_bf02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-18_18-19-50_bm04_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-18_19-35-14_bm03_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-21_18-00-15_bf04_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-21_19-14-09_bm02_k1DLC_resnet101_DevRats2Jul21shuffle1_500000'], 'subgroups': {'group1': ['2023-05-09_15-52-24_bm01_badinjxnDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-09_16-02-49_bm02_k5DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-09_16-36-04_bm03_salDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-09_17-02-19_bm04_k1DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-09_17-41-11_bf01_nal10k10DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-09_18-06-23_bf02_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-09_19-16-35_bf03_k5DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-09_20-12-22_bf04_k25DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_11-00-30_bm01_k5DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_11-30-56_bm02_salDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_12-01-26_bm03_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_13-16-30_bm04_salDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_13-44-36_bf01_cocDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_14-09-38_bf02_k1DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_16-47-52_bf03_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-15_18-18-22_bf04_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-20_17-11-22_bm01_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-20_18-17-01_bm02_cocDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_13-24-45_bm03_k25DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_13-52-07_bm04_halDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_14-22-22_bm04_halketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_14-53-10_bf01_halDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_15-23-17_bf01_halketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_16-17-57_bf02_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_17-26-26_bf03_halDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_17-58-44_bf03_halketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_18-25-03_bf04_halDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-21_18-58-40_bf04_halketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-25_13-59-51_bm01_cocDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-25_14-30-54_bm02_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-26_12-51-23_bm03_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-26_13-18-27_bm04_k5DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-26_13-47-55_bf01_salDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-26_14-27-59_bf02_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-26_14-56-26_bf03_salDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-26_15-22-23_bf04_k1DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-30_13-39-15_bm01_k25DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-30_14-08-51_bm02_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-31_11-26-43_bm04_badinjxnDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-31_11-34-17_bm03_k1DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-31_12-02-23_bf02_cocDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-31_17-08-06_bf01_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-31_18-28-16_bf03_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-05-31_19-08-34_bf04_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-09_16-55-57_bm01_halDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-09_17-33-31_bm01_halketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-09_18-18-49_bm02_k25DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-09_18-53-52_bm03_halDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-09_19-25-00_bm03_halketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-09_20-06-37_bm04_nalket_cutshortDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-10_20-30-02_bf01_k1DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-10_20-57-51_bf02_k25DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-10_21-26-14_bf03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-10_21-52-24_bf04_k5DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_17-16-18_bm01_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_17-54-08_bm02_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_18-21-57_bm03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_19-15-24_bm04_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_19-46-22_bf01_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_20-51-12_bf02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_21-28-12_bf04_salivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-15_22-01-50_bf03_salivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-20_15-03-24_bm02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-20_15-41-01_bm03_salivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-20_16-13-48_bm04_salivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-20_16-46-29_bf01_k5DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-20_17-21-51_bf02_salivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-20_17-54-10_bf03_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-20_19-10-38_bf04_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-26_15-35-45_bf01_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-26_16-15-58_bf02_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-26_16-49-06_bf03_co10DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-26_17-16-08_bf04_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-26_18-27-06_bm01_k25DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-26_19-00-05_bm02_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-27_10-14-36_bm03_co10DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-06-27_10-47-10_bm04_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_15-04-47_bm01_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_16-15-21_bm02_salivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_16-46-00_bm03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_17-54-37_bm04_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_19-20-49_bf01_salivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_19-48-04_bf02_salDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_20-27-25_bf03_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-03_20-52-16_bf04_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-08_14-12-55_bf01_k5iv_badinjxnDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-08_14-42-24_bf02_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-08_15-13-11_bf03_k25DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-08_15-47-29_bm01_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-08_16-19-27_bm02_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-08_16-55-20_bm03_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-13_14-33-00_bf03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-13_15-45-26_bm03_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-13_16-56-46_bm04_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-14_13-56-09_bm01_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-14_15-05-22_bm02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-14_16-25-47_bf01_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-14_17-04-10_bf02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-18_18-19-50_bm04_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-18_19-35-14_bm03_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-21_18-00-15_bf04_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000', '2023-07-21_19-14-09_bm02_k1DLC_resnet101_DevRats2Jul21shuffle1_500000']}, 'remappings': [[None, None], [None, None]], 'boris_directory': None, 'boris_to_pose_pairings': [[None, None], [None, None], [None, None]]}\n"
     ]
    }
   ],
   "source": [
    "config = metadata.load_project(config_path)\n",
    "print(config)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-09-17T23:40:53.675228Z",
     "start_time": "2025-09-17T23:40:53.653575Z"
    }
   },
   "id": "222245d6e4cd414b",
   "execution_count": 4
  },
  {
   "cell_type": "markdown",
   "source": [
    "Say we want to edit our config - perhaps to have subgroups for later analyses, or boris to pose pairings. We can do this by directly editing the config file in a text editor on our machine (often easier), or we can also edit the config object with and save the edited config."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "b3044045d0af208a"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dict_keys(['group1'])\n",
      "dict_keys(['sal', 'k1', 'k5', 'k10'])\n",
      "[2025-09-17 16:40:53.688855] Saved specified config object at /Users/snewbank/Behavior/MARIPoSA_demo_data//250917_test/config_PS.yaml, overwriting an existing file at that path\n"
     ]
    }
   ],
   "source": [
    "# See what the existing subgroups are\n",
    "print(config[\"subgroups\"].keys())\n",
    "\n",
    "#Make a blank slate\n",
    "config[\"subgroups\"]={}\n",
    "\n",
    "#Save our own subgroups, using paths from config[\"project_files\"]\n",
    "config[\"subgroups\"][\"sal\"]=['2023-06-15_21-28-12_bf04_salivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-06-15_22-01-50_bf03_salivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-06-20_15-41-01_bm03_salivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-06-20_16-13-48_bm04_salivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-06-20_17-21-51_bf02_salivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-07-03_16-15-21_bm02_salivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-07-03_19-20-49_bf01_salivDLC_resnet101_DevRats2Jul21shuffle1_500000']\n",
    "\n",
    "config[\"subgroups\"][\"k1\"]=['2023-06-20_19-10-38_bf04_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-06-26_15-35-45_bf01_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-06-26_19-00-05_bm02_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-06-27_10-47-10_bm04_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-07-08_14-42-24_bf02_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-07-08_15-47-29_bm01_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000']\n",
    "\n",
    "config[\"subgroups\"][\"k5\"]=['2023-06-15_17-16-18_bm01_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-06-26_16-15-58_bf02_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-07-03_20-27-25_bf03_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-07-08_16-19-27_bm02_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-07-14_16-25-47_bf01_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-07-18_19-35-14_bm03_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000']\n",
    "\n",
    "config[\"subgroups\"][\"k10\"]=['2023-05-15_16-47-52_bf03_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-05-15_18-18-22_bf04_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-05-25_14-30-54_bm02_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-05-31_17-08-06_bf01_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-07-13_15-45-26_bm03_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-07-14_13-56-09_bm01_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000',\n",
    " '2023-07-18_18-19-50_bm04_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000']\n",
    "\n",
    "#see what the new config subgroups are\n",
    "print(config[\"subgroups\"].keys())\n",
    "\n",
    "#save the new config\n",
    "metadata.save_edited_project(config,config_path)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-09-17T23:40:53.693977Z",
     "start_time": "2025-09-17T23:40:53.675826Z"
    }
   },
   "id": "4989c4c9a146f21f",
   "execution_count": 5
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 1.2 Creating a pose estimation project\n",
    "This will be a similar process, except we start with metadata.create_PE_project, since our input data may come from different sources (DeepLabCut or OpenFace) and take different formats."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "9e14620684412e01"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function create_PE_project in module utils.metadata:\n",
      "\n",
      "create_PE_project(project_name, data_directory, data_source, output_directory, fps)\n",
      "    Make project directory and write config_PE.yaml file.\n",
      "    \n",
      "    :param project_name: what to call your creation\n",
      "    :param data_directory: path to source data\n",
      "    :param data_source: DeepLabCut or SLEAP\n",
      "    :param output_directory: path where output directory and config file should be created\n",
      "    :param fps: frames per second\n",
      "    :return: path to saved config (in specified output directory)\n"
     ]
    }
   ],
   "source": [
    "help(metadata.create_PE_project)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-09-17T23:40:53.719236Z",
     "start_time": "2025-09-17T23:40:53.695177Z"
    }
   },
   "id": "ded94007aba665b9",
   "execution_count": 6
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Lforepaw', 'Lhindpaw', 'Rforepaw', 'Rhindpaw', 'bodyparts', 'nose', 'tailbase']\n",
      "[2025-09-17 16:40:53.739740] Made project 250917_test_PE and saved at /Users/snewbank/Behavior/MARIPoSA_demo_data//250917_test_PE/config_PE.yaml\n"
     ]
    }
   ],
   "source": [
    "project_name=\"test_PE\"\n",
    "data_directory=demo_dir+\"PE-TEST-25-05-20-rat-DLC/\"\n",
    "data_source=\"DeepLabCut\"\n",
    "output_directory=demo_dir\n",
    "fps=30\n",
    "\n",
    "PE_config_path = metadata.create_PE_project(project_name,\n",
    "                                            data_directory,\n",
    "                                            data_source,\n",
    "                                            output_directory,\n",
    "                                            fps)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-09-17T23:40:53.747984Z",
     "start_time": "2025-09-17T23:40:53.700571Z"
    }
   },
   "id": "f067e15e0ab97f25",
   "execution_count": 7
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now we can load our pose estimation config (and edit it, if we wish) just the same as if it were a pose segmentation config."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "7f36a60b9f50ae7b"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'project_name': '250917_test_PE', 'data_directory': '/Users/snewbank/Behavior/MARIPoSA_demo_data/PE-TEST-25-05-20-rat-DLC/', 'project_directory': '/Users/snewbank/Behavior/MARIPoSA_demo_data//250917_test_PE', 'data_type': 'Pose estimation', 'data_source': 'DeepLabCut', 'fps': '30', 'keypoints': ['Lforepaw', 'Lhindpaw', 'Rforepaw', 'Rhindpaw', 'bodyparts', 'nose', 'tailbase'], 'project_files': ['2023-05-09_15-52-24_bm01_badinjxnDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-09_16-02-49_bm02_k5DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-09_16-36-04_bm03_salDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-09_17-02-19_bm04_k1DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-09_17-41-11_bf01_nal10k10DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-09_18-06-23_bf02_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-09_19-16-35_bf03_k5DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-09_20-12-22_bf04_k25DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_11-00-30_bm01_k5DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_11-30-56_bm02_salDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_12-01-26_bm03_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_13-16-30_bm04_salDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_13-44-36_bf01_cocDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_14-09-38_bf02_k1DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_16-47-52_bf03_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_18-18-22_bf04_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-20_17-11-22_bm01_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-20_18-17-01_bm02_cocDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_13-24-45_bm03_k25DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_13-52-07_bm04_halDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_14-22-22_bm04_halketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_14-53-10_bf01_halDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_15-23-17_bf01_halketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_16-17-57_bf02_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_17-26-26_bf03_halDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_17-58-44_bf03_halketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_18-25-03_bf04_halDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_18-58-40_bf04_halketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-25_13-59-51_bm01_cocDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-25_14-30-54_bm02_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-26_12-51-23_bm03_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-26_13-18-27_bm04_k5DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-26_13-47-55_bf01_salDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-26_14-27-59_bf02_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-26_14-56-26_bf03_salDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-26_15-22-23_bf04_k1DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-30_13-39-15_bm01_k25DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-30_14-08-51_bm02_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-31_11-26-43_bm04_badinjxnDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-31_11-34-17_bm03_k1DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-31_12-02-23_bf02_cocDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-31_17-08-06_bf01_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-31_18-28-16_bf03_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-31_19-08-34_bf04_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-09_16-55-57_bm01_halDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-09_17-33-31_bm01_halketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-09_18-18-49_bm02_k25DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-09_18-53-52_bm03_halDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-09_19-25-00_bm03_halketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-09_20-06-37_bm04_nalket_cutshortDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-10_20-30-02_bf01_k1DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-10_20-57-51_bf02_k25DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-10_21-26-14_bf03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-10_21-52-24_bf04_k5DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_17-16-18_bm01_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_17-54-08_bm02_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_18-21-57_bm03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_19-15-24_bm04_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_19-46-22_bf01_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_20-51-12_bf02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_21-28-12_bf04_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_22-01-50_bf03_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-20_15-03-24_bm02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-20_15-41-01_bm03_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-20_16-13-48_bm04_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-20_16-46-29_bf01_k5DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-20_17-21-51_bf02_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-20_17-54-10_bf03_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-20_19-10-38_bf04_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-26_15-35-45_bf01_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-26_16-15-58_bf02_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-26_16-49-06_bf03_co10DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-26_17-16-08_bf04_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-26_18-27-06_bm01_k25DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-26_19-00-05_bm02_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-27_10-14-36_bm03_co10DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-27_10-47-10_bm04_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_15-04-47_bm01_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_16-15-21_bm02_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_16-46-00_bm03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_17-54-37_bm04_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_19-20-49_bf01_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_19-48-04_bf02_salDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_20-27-25_bf03_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_20-52-16_bf04_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-08_14-12-55_bf01_k5iv_badinjxnDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-08_14-42-24_bf02_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-08_15-13-11_bf03_k25DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-08_15-47-29_bm01_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-08_16-19-27_bm02_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-08_16-55-20_bm03_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-13_14-33-00_bf03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-13_15-45-26_bm03_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-13_16-56-46_bm04_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-14_13-56-09_bm01_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-14_15-05-22_bm02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-14_16-25-47_bf01_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-14_17-04-10_bf02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-18_18-19-50_bm04_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-18_19-35-14_bm03_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-21_18-00-15_bf04_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-21_19-14-09_bm02_k1DLC_resnet101_DevRats2Jul21shuffle1_500000.csv'], 'subgroups': {'group1': ['2023-05-09_15-52-24_bm01_badinjxnDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-09_16-02-49_bm02_k5DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-09_16-36-04_bm03_salDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-09_17-02-19_bm04_k1DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-09_17-41-11_bf01_nal10k10DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-09_18-06-23_bf02_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-09_19-16-35_bf03_k5DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-09_20-12-22_bf04_k25DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_11-00-30_bm01_k5DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_11-30-56_bm02_salDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_12-01-26_bm03_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_13-16-30_bm04_salDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_13-44-36_bf01_cocDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_14-09-38_bf02_k1DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_16-47-52_bf03_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-15_18-18-22_bf04_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-20_17-11-22_bm01_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-20_18-17-01_bm02_cocDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_13-24-45_bm03_k25DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_13-52-07_bm04_halDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_14-22-22_bm04_halketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_14-53-10_bf01_halDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_15-23-17_bf01_halketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_16-17-57_bf02_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_17-26-26_bf03_halDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_17-58-44_bf03_halketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_18-25-03_bf04_halDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-21_18-58-40_bf04_halketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-25_13-59-51_bm01_cocDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-25_14-30-54_bm02_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-26_12-51-23_bm03_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-26_13-18-27_bm04_k5DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-26_13-47-55_bf01_salDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-26_14-27-59_bf02_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-26_14-56-26_bf03_salDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-26_15-22-23_bf04_k1DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-30_13-39-15_bm01_k25DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-30_14-08-51_bm02_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-31_11-26-43_bm04_badinjxnDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-31_11-34-17_bm03_k1DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-31_12-02-23_bf02_cocDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-31_17-08-06_bf01_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-31_18-28-16_bf03_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-05-31_19-08-34_bf04_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-09_16-55-57_bm01_halDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-09_17-33-31_bm01_halketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-09_18-18-49_bm02_k25DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-09_18-53-52_bm03_halDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-09_19-25-00_bm03_halketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-09_20-06-37_bm04_nalket_cutshortDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-10_20-30-02_bf01_k1DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-10_20-57-51_bf02_k25DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-10_21-26-14_bf03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-10_21-52-24_bf04_k5DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_17-16-18_bm01_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_17-54-08_bm02_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_18-21-57_bm03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_19-15-24_bm04_nalketDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_19-46-22_bf01_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_20-51-12_bf02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_21-28-12_bf04_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-15_22-01-50_bf03_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-20_15-03-24_bm02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-20_15-41-01_bm03_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-20_16-13-48_bm04_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-20_16-46-29_bf01_k5DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-20_17-21-51_bf02_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-20_17-54-10_bf03_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-20_19-10-38_bf04_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-26_15-35-45_bf01_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-26_16-15-58_bf02_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-26_16-49-06_bf03_co10DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-26_17-16-08_bf04_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-26_18-27-06_bm01_k25DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-26_19-00-05_bm02_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-27_10-14-36_bm03_co10DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-06-27_10-47-10_bm04_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_15-04-47_bm01_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_16-15-21_bm02_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_16-46-00_bm03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_17-54-37_bm04_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_19-20-49_bf01_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_19-48-04_bf02_salDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_20-27-25_bf03_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-03_20-52-16_bf04_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-08_14-12-55_bf01_k5iv_badinjxnDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-08_14-42-24_bf02_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-08_15-13-11_bf03_k25DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-08_15-47-29_bm01_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-08_16-19-27_bm02_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-08_16-55-20_bm03_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-13_14-33-00_bf03_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-13_15-45-26_bm03_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-13_16-56-46_bm04_k10ipDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-14_13-56-09_bm01_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-14_15-05-22_bm02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-14_16-25-47_bf01_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-14_17-04-10_bf02_mk801DLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-18_18-19-50_bm04_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-18_19-35-14_bm03_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-21_18-00-15_bf04_k10scDLC_resnet101_DevRats2Jul21shuffle1_500000.csv', '2023-07-21_19-14-09_bm02_k1DLC_resnet101_DevRats2Jul21shuffle1_500000.csv']}}\n"
     ]
    }
   ],
   "source": [
    "PE_config = metadata.load_project(PE_config_path)\n",
    "print(PE_config)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-09-17T23:40:53.777830Z",
     "start_time": "2025-09-17T23:40:53.744841Z"
    }
   },
   "id": "1a7a195e9ad793c1",
   "execution_count": 8
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dict_keys(['group1'])\n",
      "dict_keys(['sal', 'k1', 'k5', 'k10'])\n",
      "[2025-09-17 16:40:53.788650] Saved specified config object at /Users/snewbank/Behavior/MARIPoSA_demo_data//250917_test_PE/config_PE.yaml, overwriting an existing file at that path\n"
     ]
    }
   ],
   "source": [
    "# See what the existing subgroups are\n",
    "print(PE_config[\"subgroups\"].keys())\n",
    "\n",
    "#Make a blank slate\n",
    "PE_config[\"subgroups\"]={}\n",
    "\n",
    "#Save our own subgroups, using paths from PE_config[\"project_files\"]\n",
    "PE_config[\"subgroups\"][\"sal\"]=['2023-06-15_21-28-12_bf04_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-06-15_22-01-50_bf03_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-06-20_15-41-01_bm03_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-06-20_16-13-48_bm04_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-06-20_17-21-51_bf02_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-07-03_16-15-21_bm02_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-07-03_19-20-49_bf01_salivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv']\n",
    "\n",
    "PE_config[\"subgroups\"][\"k1\"]=['2023-06-20_19-10-38_bf04_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-06-26_15-35-45_bf01_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-06-26_19-00-05_bm02_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-06-27_10-47-10_bm04_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-07-08_14-42-24_bf02_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-07-08_15-47-29_bm01_k1ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv']\n",
    "\n",
    "PE_config[\"subgroups\"][\"k5\"]=['2023-06-15_17-16-18_bm01_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-06-26_16-15-58_bf02_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-07-03_20-27-25_bf03_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-07-08_16-19-27_bm02_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-07-14_16-25-47_bf01_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-07-18_19-35-14_bm03_k5ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv']\n",
    "\n",
    "PE_config[\"subgroups\"][\"k10\"]=['2023-05-15_16-47-52_bf03_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-05-15_18-18-22_bf04_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-05-25_14-30-54_bm02_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-05-31_17-08-06_bf01_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-07-13_15-45-26_bm03_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-07-14_13-56-09_bm01_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv',\n",
    " '2023-07-18_18-19-50_bm04_k10ivDLC_resnet101_DevRats2Jul21shuffle1_500000.csv']\n",
    "\n",
    "#see what the new config subgroups are\n",
    "print(PE_config[\"subgroups\"].keys())\n",
    "\n",
    "#save the new config\n",
    "metadata.save_edited_project(PE_config,PE_config_path)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-09-17T23:40:53.808935Z",
     "start_time": "2025-09-17T23:40:53.768118Z"
    }
   },
   "id": "9f5d46f5d4cf340b",
   "execution_count": 9
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}