Config objects in vipdopt

vipdopt provides functionality for loading data from configuration files for easy access within a script. Data can be loaded from a YAML or JSON file and will be stored in the Config class.

# imports
from pathlib import Path
import sys
import yaml

import numpy as np
import matplotlib.pyplot as plt

np.set_printoptions(threshold=100)

# Get vipdopt directory path from Notebook
parent_dir = str(Path().resolve().parents[2])

# Add to sys.path
sys.path.insert(0, parent_dir)

# Imports from vipdopt
from vipdopt.configuration import Config, TemplateRenderer, SonyBayerRenderer, SonyBayerConfig
from vipdopt.simulation import Power
from vipdopt.utils import rmtree

The Config class is essentially a Python dict with keys for all of the values in the original configuration file.

# Load file
cfg = Config()
cfg.read_file('config_example_3d.yml')

# Can also create a `Config` directly from a file
cfg2 = Config.from_file('config_example_3d.yml')

assert cfg == cfg2

# Accessing properties using `dict`-like access
print(f'At first, device_scale_um = {cfg["device_scale_um"]}')
cfg['device_scale_um'] = 0.06
print(f'Now, device_scale_um = {cfg["device_scale_um"]}')

At first, device_scale_um = 0.051
Now, device_scale_um = 0.06

Config objects are particularly helpful for setting up simulation objects automatically. In the example below, a Power monitor is created using values from a Config we loaded.

cfg = Config.from_file('example_render.yml')

pow = Power('focal_monitor_0')
properites = {
    'monitor type': 'point',
    'x': cfg['adjoint_x_positions_um'][0] * 1e-6,
    'y': cfg['adjoint_y_positions_um'][0] * 1e-6,
    'z': cfg['adjoint_vertical_um'] * 1e-6,
    'override global monitor settings': 1,
    'use wavelength spacing': 1,
    'use source limits': 1,
    'frequency points': cfg['num_design_frequency_points'],
}
pow.update(**properites)

Generating Configuration Files using Jinja2

Sometimes you may wish to compute certain properties using the values of other ones in your configuration file. For example pixel_width = 2 and num_pixels = 10 and you want to have a third value total_width = pixel_width * num_pixels. This would save time if you later wanted to tweak the values in your configuration file, as you would only need to change pixel_width and num_pixels rather than all of three. Unfortunately, most standard configuration file formats do not support this kind of functionality.

Jinja2 is an extensible templating engine. Special placeholders are placed in a template file to allow writing code similar to Python syntax. Then data is passed to the template to compute the placeholder values and render a final document.

For the total_width example, this template would look something like:

pixel_width: {{ data.pixel_width }}
num_pixels: {{ data.num_pixels }}
total_width: {{ data.pixel_width * data.num_pixels }}

Here, data is a dictionary being passed into the template renderer, which allows the use of its various values. the double curly braces {{}} serve as the placeholders that are evaluated by the renderer.

If we were to pass a dictionary such as {'pixel_wdith': 2, 'num_pixels': 10} to the renderer, our output would be:

pixel_width: 2
num_pixels: 10
total_width: 20

For more information on Jinja2, please check the official documentation. This guide also serves as a good starting point for formatting template files.

General Workflow using Jinja2

The convenience of Jinja2 creates a sort of workflow one should use when using configuration files in vipdopt:

1. Create an initial configuration file with general values (e.g. pixel_width)

2. Create a template file that uses the values from the configuration file to compute other values

3. Render the template file into a final “rendered” configuration file

Below are two examples of this workflow in code.

#
# Pixel Example
#

pixel_source_directory = Path('pixel_example/')
pixel_source_directory.mkdir(exist_ok=True)

# Step 1 - Create initial config file
initial_data = {
    'pixel_width': 2,
    'num_pixels': 10,
}
config_file = pixel_source_directory / 'initial_config.yaml'
with config_file.open('w') as f:
    yaml.safe_dump(initial_data, f)

# Step 2 - Create template file
template_str = """pixel_width: {{ data.pixel_width }}
num_pixels: {{ data.num_pixels }}
total_width: {{ data.pixel_width * data.num_pixels }}
"""
template_file = pixel_source_directory / 'template.j2'
with template_file.open('w') as f:
    f.write(template_str)

# Step 3 - Render final config file
loaded_data = Config.from_file(config_file)
renderer = TemplateRenderer(pixel_source_directory)
renderer.set_template(template_file)
output_str = renderer.render(data=loaded_data)

print(output_str)

pixel_width: 2
num_pixels: 10
total_width: 20

#
# SONY Bayer Filter Example
#
pixel_source_directory = Path('.')

# Step 1 - create initial config file (already done)
og_config_file = pixel_source_directory / 'config_example_3d.yml'
data = SonyBayerConfig.from_file(og_config_file)

# Step 2 - create template file (already done)
template_filename =  'derived_simulation_properties.j2'

# Step 3 - render template file
renderer = SonyBayerRenderer(pixel_source_directory)
renderer.set_template(template_filename)

output_file = pixel_source_directory / 'test_render.yml'

renderer.render_to_file(output_file, data=data, pi=np.pi)