Managing Projects in vipdopt

vipdopt includes a Project class for organizing all of your simulations, output data, and plots in one project directory. The Project class also enables saving the progress of an optimization and restarting from that checkpoint.

# imports
from pathlib import Path
import sys

import numpy as np
import matplotlib.pyplot as plt

np.set_printoptions(threshold=100)

# Get vipdopt directory path from Notebook
parent_dir = str(Path().resolve().parents[2])

# Add to sys.path
sys.path.insert(0, parent_dir)

# Imports from vipdopt
from vipdopt.optimization import Device, Sigmoid, Scale, LumericalOptimization, BayerFilterFoM, AdamOptimizer, SuperFoM
from vipdopt.simulation import LumericalSimulation, GaussianSource, DipoleSource, Power, Profile
from vipdopt.configuration import SonyBayerConfig

from vipdopt.project import Project, create_internal_folder_structure

When you initialize a new Project it will be mostly empty. You’ll need to set all of the parts (base simulation, optimization, optimizer, etc.) manually.

Throughout this notebook, we will be using the “Sony Color Router” design that’s appeared in the other tutorial notebooks.

# Creating all of the parts first
cfg = SonyBayerConfig()
cfg.read_file('config_example_3d.yml')
base_sim = LumericalSimulation('simulation_example.json')
optimizer = AdamOptimizer(cfg['fixed_step_size'], (0.9, 0.999))

coords = {
    'x': np.linspace(0, 2.04e-6, 60),
    'y': np.linspace(0, 2.04e-6, 60),
    'z': np.linspace(0, 2.04e-6, 60),
}
lambda_vector = np.linspace(
    cfg['lambda_min_um'],
    cfg['lambda_max_um'],
    cfg['num_bands'] * cfg['num_points_per_band']
)
n_freq = len(lambda_vector)

device = Device(
    (60, 60, 60),
    (0, 1),
    coords,
    'color_router',
    randomize=True,
    init_seed=23,
    filters=[Sigmoid(0.05, 0.1)]
)

foms = [
    BayerFilterFoM(
        'TE',
        [GaussianSource(f'forward_src_{axis}')],
        [GaussianSource(f'forward_src_{axis}'), DipoleSource(f'adjoint_src_{n}{axis}')],
        [
            Power(f'focal_monitor_{n}'),
            Power(f'transmission_monitor_{n}'),
            Profile('design_efield_monitor')
        ],
        [Profile('design_efield_monitor')],
        range(n_freq),
        [],
        all_freqs=lambda_vector
    ) for axis in 'xy' for n in range(4)
]
weights = [1] * len(foms)
combined_fom = SuperFoM(((f,) for f in foms), weights)

opt = LumericalOptimization(
    base_sim,
    device,
    optimizer,
    combined_fom,
    epoch_list=np.linspace(
        cfg['iter_per_epoch'],
        cfg['iter_per_epoch'] * cfg['max_epochs'],
        cfg['max_epochs'], dtype=int
    ),
)

Now that the parts are made we can make our Project manually.

project = Project()
project.config = cfg
project.optimization = opt
project.optimizer = optimizer
project.base_sim = base_sim
project.foms = foms
project.weights = weights

A Project has a specific folder structure. This structure can be created automatically using create_internal_structure. Below is what the internal structure looks like:

├───.tmp
├───data
│   ├───checkpoints
│   ├───opt_info
│   │   └───plots
│   └───saved_scripts
├───device
└───eval
    ├───configs
    └───utils

• .tmp stores intermediate files and completed simulation data

• data/checkpoints contains checkpoints for optimiztion progress

• data/opt_info contains information about the optimization

• data/opt_info/plots contains plots generated during the optimization

• data/saved_scripts contains scripts that are included as part of the optimization

• device contains saved device parameters over each iteration

• eval contains configurations and utility functions for evaluating the optimization

# Let's use the directory "color_router_proj" as our project folder
proj_dir = Path('color_router_proj')
project.subdirectories = create_internal_folder_structure(proj_dir)

project.save_as(proj_dir)

If we have a previously saved project folder, we can load our project into code for faster setup.

project.load_project(proj_dir)