In this R Notebook we preprocess spatial and corresponding reference scRNA-seq data of mouse olfactory bulb (MOB) for cell type deconvolution.

  1. Spatial data preprocessing:

    1.1 Input original data files

    1.2 Output data files for cell type deconvolution

  2. Reference scRNA-seq data preprocessing:

    2.1 Input original data files

    scRNA-seq data are downloaded from GSE121891.

    2.2 Output data files for cell type deconvolution

1 Version

version[['version.string']]
[1] "R version 4.2.2 Patched (2022-11-10 r83330)"

2 Preprocess MOB spatial dataset

2.1 Read original data file Rep12_MOB_count_matrix-1.tsv

file_name = file.path(home.dir, 'Rep12_MOB_count_matrix-1.tsv')
org_data = read.csv(file_name, sep = '\t', check.names = F, header = T, row.names = 1)
print(sprintf('load data from %s', file_name))
[1] "load data from /home/hill103/Documents/SharedFolder/ToHost/CVAE-GLRM_Analysis/RealData/MOB/Rep12_MOB_count_matrix-1.tsv"
print(sprintf('spots: %d; genes: %d', nrow(org_data), ncol(org_data)))
[1] "spots: 282; genes: 16034"
org_data[1:5, 1:5]

2.2 Save files for deconvolution

2.2.1 Spatial spot nUMI

No filtering on spots or genes, directly save all spots and genes into file MOB_spatial_spot_nUMI.csv. Rows as spatial spots and columns as genes.

write.csv(org_data, 'MOB_spatial_spot_nUMI.csv')
Registered S3 method overwritten by 'data.table':
  method           from
  print.data.table     
print(sprintf('save %d gene nUMIs of %d spatial spots into file %s', ncol(org_data), nrow(org_data), 'MOB_spatial_spot_nUMI.csv'))
[1] "save 16034 gene nUMIs of 282 spatial spots into file MOB_spatial_spot_nUMI.csv"

2.2.2 Physical Locations of spatial spots

Directly extract the spatial x and y coordinates from spot names, followed by rounding to nearest integers, then saved into file MOB_spatial_spot_loc.csv.

local_df = data.frame(names = row.names(org_data), row.names = row.names(org_data))
local_df = local_df %>%
  tidyr::separate_wider_delim(names, 'x', names = c('x', 'y'))
local_df = as.data.frame(local_df)
row.names(local_df) = row.names(org_data)

local_df['x'] = round(as.numeric(local_df$x))
local_df['y'] = round(as.numeric(local_df$y))

local_df[1:5, ]

write.csv(local_df, 'MOB_spatial_spot_loc.csv')
print(sprintf('save Physical Locations of spatial spots into file %s', 'MOB_spatial_spot_loc.csv'))
[1] "save Physical Locations of spatial spots into file MOB_spatial_spot_loc.csv"

2.2.3 Adjacency Matrix of spatial spots

We define the neighborhood of a spatial spot contains the adjacent left, right, top and bottom spot, that is, one spot has at most 4 neighbors.

The generated Adjacency Matrix A only contains 1 and 0, where 1 represents corresponding two spots are adjacent spots according to the definition of neighborhood, while value 0 for non-adjacent spots. Note all diagonal entries are 0s.

Adjacency Matrix are saved into file MOB_spatial_spot_adjacency_matrix.csv.

getNeighbour = function(array_row, array_col) {
  # based on the (row, col) of one spot, return the (row, col) of all 4 neighbours
  return(list(c(array_row-1, array_col),
              c(array_row+1, array_col),
              c(array_row+0, array_col-1),
              c(array_row+0, array_col+1)))
}

# adjacency matrix
A = matrix(0, nrow = nrow(local_df), ncol = nrow(local_df))
row.names(A) = rownames(local_df)
colnames(A) = rownames(local_df)
for (i in 1:nrow(local_df)) {
  barcode = rownames(local_df)[i]
  array_row = local_df[i, 'y']
  array_col = local_df[i, 'x']
  
  # get neighbors
  neighbours = getNeighbour(array_row, array_col)
  
  # fill the adjacency matrix
  for (this.vec in neighbours) {
    tmp.p = rownames(local_df[local_df$y==this.vec[1] & local_df$x==this.vec[2], ])
    
    if (length(tmp.p) >= 1) {
      # target spots have neighbors in selected spots
      for (neigh.barcode in tmp.p) {
        A[barcode, neigh.barcode] = 1
      }
    }
  }
}

A[1:5, 1:5]
              16.918x16.996 18.017x17.034 20.075x17.059 18.979x17.065 21.937x16.967
16.918x16.996             0             1             0             0             0
18.017x17.034             1             0             0             1             0
20.075x17.059             0             0             0             1             0
18.979x17.065             0             1             1             0             0
21.937x16.967             0             0             0             0             0
write.csv(A, 'MOB_spatial_spot_adjacency_matrix.csv')
print(sprintf('save Adjacency Matrix of spatial spots into file %s', 'MOB_spatial_spot_adjacency_matrix.csv'))
[1] "save Adjacency Matrix of spatial spots into file MOB_spatial_spot_adjacency_matrix.csv"

Plot Adjacency Matrix. Each node is spot, spots within neighborhood are connected with edges.

g = graph_from_adjacency_matrix(A, 'undirected', add.colnames = NA, add.rownames = NA)
# manually set nodes x and y coordinates
vertex_attr(g, name = 'x') = local_df$x
vertex_attr(g, name = 'y') = local_df$y
plot(g, vertex.size=5, edge.width=4, margin=-0.05)

3 Proprocess reference scRNA-seq data

3.1 Read and preprocess scRNA-seq meta data

Original meta data file is GSE121891_Figure_2_metadata.txt.gz downloaded from GSE121891. It contains meta data of 21,746 Neurons cells from mouse olfactory bulb. The cell type annotation is stored in column FinalIds, which includes total 18 distinct annotations. We selected 5 cell types and combine the subtypes as below:

  1. granule cells (GC): “n03-GC-1” + “n07-GC-2” + “n09-GC-3” + “n10-GC-4” + “n11-GC-5” + “n12-GC-6” + “n14-GC-7”
  2. olfactory sensory neurons (OSNs): “n01-OSNs”
  3. periglomerular cells (PGC): “n02-PGC-1” + “n05-PGC-2” + “n08-PGC-3”
  4. mitral and tufted cells (M/TC): “n15-M/TC-1” + “n16-M/TC-2” + “n17-M/TC-3”
  5. external plexiform layer interneurons (EPL-IN): “n18-EPL-IN”

3 Subtypes “n04-Immature”, “n06-Transition” and “n13-AstrocyteLike” are discarded.

NO further filtering on cells, i.e. all 12,801 cells of these 5 selected cell types will be used for cell type deconvolution.

file_name = file.path(home.dir, 'GSE121891_Figure_2_metadata.txt.gz')
ref_meta = read.csv(gzfile(file_name), sep='\t', check.names = F, header = T, row.names = 1)
print(sprintf('load data from %s', file_name))
[1] "load data from /home/hill103/Documents/SharedFolder/ToHost/CVAE-GLRM_Analysis/RealData/MOB/GSE121891_Figure_2_metadata.txt.gz"
print(sprintf('total %d cells with distinct %d cell type annotations', nrow(ref_meta), length(unique(ref_meta$FinalIds))))
[1] "total 21746 cells with distinct 18 cell type annotations"
# remove unwanted 3 subtypes
ref_meta = ref_meta[ref_meta$FinalIds %notin% c("n04-Immature", "n06-Transition", "n13-AstrocyteLike"), ]
print(sprintf('remove 3 cell subtype annotations, remain %d cells', nrow(ref_meta)))
[1] "remove 3 cell subtype annotations, remain 12801 cells"
# combine subtypes
ref_meta$celltype = ""
ref_meta[ref_meta$FinalIds %in% c("n03-GC-1", "n07-GC-2", "n09-GC-3", "n10-GC-4", "n11-GC-5", "n12-GC-6", "n14-GC-7"), "celltype"] = "GC"
ref_meta[ref_meta$FinalIds == "n01-OSNs", "celltype"] = "OSNs"
ref_meta[ref_meta$FinalIds %in% c("n02-PGC-1", "n05-PGC-2", "n08-PGC-3"), "celltype"] = "PGC"
ref_meta[ref_meta$FinalIds %in% c("n15-M/TC-1", "n16-M/TC-2", "n17-M/TC-3"), "celltype"] = "M/TC"
ref_meta[ref_meta$FinalIds == "n18-EPL-IN", "celltype"] = "EPL-IN"

table(ref_meta$celltype)

EPL-IN     GC   M/TC   OSNs    PGC 
   161   8614   1133   1200   1693 
ref_meta[1:5, c('FinalIds', 'celltype')]

Save cell type annotation to file MOB_ref_scRNA_cell_celltype.csv

write.csv(ref_meta[, 'celltype', drop=F], 'MOB_ref_scRNA_cell_celltype.csv')
print(sprintf('save cell type annotation of reference scRNA-seq cells into file %s', 'MOB_ref_scRNA_cell_celltype.csv'))
[1] "save cell type annotation of reference scRNA-seq cells into file MOB_ref_scRNA_cell_celltype.csv"

3.2 Read and preprocess scRNA-seq nUMI data

Original gene nUMI count data file is GSE121891_OB_6_runs.raw.dge.csv.gz downloaded from GSE121891. It contains total 52,549 cells and 18,560 genes.

We just selected 12,801 cells of the 5 selected cell types by barcodes, and discard other cells. NO filtering on genes, i.e. all 18,560 genes will be used for cell type deconvolution.

file_name = file.path(home.dir, 'GSE121891_OB_6_runs.raw.dge.csv.gz')
ref_data = data.table::fread(file_name, sep = ",", check.names = FALSE, select = c('V1', row.names(ref_meta)))
gene_names = ref_data$V1

# transpose it
ref_data = as.data.frame(data.table::transpose(ref_data %>%
  select(row.names(ref_meta))))

row.names(ref_data) = row.names(ref_meta)
colnames(ref_data) = gene_names

print(sprintf('load data from %s', file_name))
[1] "load data from /home/hill103/Documents/SharedFolder/ToHost/CVAE-GLRM_Analysis/RealData/MOB/GSE121891_OB_6_runs.raw.dge.csv.gz"
print(sprintf('cells: %d; genes: %d', nrow(ref_data), ncol(ref_data)))
[1] "cells: 12801; genes: 18560"
ref_data[1:5, 1:5]

Save scRNA-seq nUMI matrix to file MOB_ref_scRNA_cell_nUMI.csv.gz

data.table::fwrite(ref_data, 'MOB_ref_scRNA_cell_nUMI.csv.gz', row.names = T)
print(sprintf('save nUMI matrix of reference scRNA-seq cells into gzip compressed file %s', 'MOB_ref_scRNA_cell_nUMI.csv.gz'))
[1] "save nUMI matrix of reference scRNA-seq cells into gzip compressed file MOB_ref_scRNA_cell_nUMI.csv.gz"
LS0tCnRpdGxlOiAiUHJlcHJvY2VzcyBNT0IgZGF0YSBmb3IgY2VsbCB0eXBlIGRlY29udm9sdXRpb24iCmF1dGhvcjogIk5pbmdzaGFuIExpICYgSmlheWkgWmhhbyIKZGF0ZTogIjIwMjMvMDMvMTkiCm91dHB1dDogCiAgaHRtbF9ub3RlYm9vazoKICAgIGNvZGVfZm9sZGluZzogaGlkZQogICAgaGlnaGxpZ2h0OiB0YW5nbwogICAgbnVtYmVyX3NlY3Rpb25zOiB5ZXMKICAgIHRoZW1lOiB1bml0ZWQKICAgIHRvYzogeWVzCiAgICB0b2NfZGVwdGg6IDYKICAgIHRvY19mbG9hdDogeWVzCi0tLQoKYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gVFJVRSwgZXZhbCA9IFRSVUUsIHdhcm5pbmcgPSBGQUxTRSwgbWVzc2FnZSA9IEZBTFNFLCByZXN1bHRzPSdob2xkJywgZmlnLndpZHRoID0gNywgZmlnLmhlaWdodCA9IDUsIGRwaSA9IDMwMCkKCgpsaWJyYXJ5KGRwbHlyKQpsaWJyYXJ5KGdncGxvdDIpCmxpYnJhcnkoaWdyYXBoKQoKYCVub3RpbiVgID0gTmVnYXRlKGAlaW4lYCkKCnNldC5zZWVkKDEpCgpob21lLmRpciA9ICcvaG9tZS9oaWxsMTAzL0RvY3VtZW50cy9TaGFyZWRGb2xkZXIvVG9Ib3N0L0NWQUUtR0xSTV9BbmFseXNpcy9SZWFsRGF0YS9NT0InCgoKbXkuZGlzdGluY3QuY29sb3JzMjAgPSBjKCIjZTYxOTRiIiwgIiMzY2I0NGIiLCAiI2ZmZTExOSIsICIjNDM2M2Q4IiwgIiNmNTgyMzEiLCAiIzkxMWViNCIsICIjNDZmMGYwIiwgIiNmMDMyZTYiLCAiI2JjZjYwYyIsICIjZmFiZWJlIiwgIiMwMDgwODAiLCAiIzlhNjMyNCIsICIjODAwMDAwIiwgIiNhYWZmYzMiLCAiIzgwODAwMCIsICIjMDAwMDc1IiwgIiM4MDgwODAiLCAiI2U2YmVmZiIsICIjZmZkOGIxIiwgIiMwMDAwMDAiKQoKbXkuZGlzdGluY3QuY29sb3JzNDAgPSBjKCIjMDBmZjAwIiwiI2ZmNDUwMCIsIiMwMGNlZDEiLCIjNTU2YjJmIiwiI2EwNTIyZCIsIiM4YjAwMDAiLCIjODA4MDAwIiwiIzQ4M2Q4YiIsIiMwMDgwMDAiLCIjMDA4MDgwIiwiIzQ2ODJiNCIsIiMwMDAwODAiLCIjOWFjZDMyIiwiI2RhYTUyMCIsIiM3ZjAwN2YiLCIjOGZiYzhmIiwiI2IwMzA2MCIsIiNkMmI0OGMiLCIjNjk2OTY5IiwiI2ZmOGMwMCIsIiMwMGZmN2YiLCIjZGMxNDNjIiwiI2Y0YTQ2MCIsIiMwMDAwZmYiLCIjYTAyMGYwIiwiI2FkZmYyZiIsIiNmZjAwZmYiLCIjMWU5MGZmIiwiI2YwZTY4YyIsIiNmYTgwNzIiLCIjZmZmZjU0IiwiI2RkYTBkZCIsIiM4N2NlZWIiLCIjN2I2OGVlIiwiI2VlODJlZSIsIiM5OGZiOTgiLCIjN2ZmZmQ0IiwiI2ZmYjZjMSIsIiNkY2RjZGMiLCIjMDAwMDAwIikKYGBgCgoKSW4gdGhpcyBSIE5vdGVib29rIHdlIHByZXByb2Nlc3Mgc3BhdGlhbCBhbmQgY29ycmVzcG9uZGluZyByZWZlcmVuY2Ugc2NSTkEtc2VxIGRhdGEgb2YgbW91c2Ugb2xmYWN0b3J5IGJ1bGIgKCoqTU9CKiopIGZvciBjZWxsIHR5cGUgZGVjb252b2x1dGlvbi4KCjEuICoqU3BhdGlhbCBkYXRhIHByZXByb2Nlc3NpbmcqKjoKCiAgICAxLjEgSW5wdXQgb3JpZ2luYWwgZGF0YSBmaWxlcwogICAgCiAgICAqIFJhdyBuVU1JIG9mIHNwYXRpYWwgc3BvdHM6IFtSZXAxMl9NT0JfY291bnRfbWF0cml4LTEudHN2XShodHRwczovL3d3dy5zcGF0aWFscmVzZWFyY2gub3JnL3dwLWNvbnRlbnQvdXBsb2Fkcy8yMDE2LzA3L1JlcDEyX01PQl9jb3VudF9tYXRyaXgtMS50c3YpLCBkb3dubG9hZGVkIGZyb20gW1NwYXRpYWwgUmVzZWFyY2ggd2Vic2l0ZV0oaHR0cHM6Ly93d3cuc3BhdGlhbHJlc2VhcmNoLm9yZy9yZXNvdXJjZXMtcHVibGlzaGVkLWRhdGFzZXRzL2RvaS0xMC0xMTI2c2NpZW5jZS1hYWYyNDAzLyksIGFuZCBzYW1wbGUgKipNT0IgUmVwbGljYXRlIDEyKiogaXMgc2VsZWN0ZWQgZm9yIGFuYWx5c2lzLgogICAgCiAgICAxLjIgT3V0cHV0IGRhdGEgZmlsZXMgZm9yIGNlbGwgdHlwZSBkZWNvbnZvbHV0aW9uCiAgICAKICAgICogUmF3IG5VTUkgb2Ygc3BhdGlhbCBzcG90czogW01PQl9zcGF0aWFsX3Nwb3RfblVNSS5jc3ZdKGh0dHBzOi8vZ2l0aHViLmNvbS9hejdqaDIvU0RlUEVSX0FuYWx5c2lzL2Jsb2IvbWFpbi9SZWFsRGF0YS9NT0IvTU9CX3NwYXRpYWxfc3BvdF9uVU1JLmNzdikuICoqTm8gZmlsdGVyaW5nIG9uIHNwb3RzIG9yIGdlbmVzKiosIGkuZS4gYWxsIHNwb3RzIGFuZCBnZW5lcyBhcmUgcHJlc2VydmVkLgogICAgKiBQaHlzaWNhbCBsb2NhdGlvbiBvZiBzcGF0aWFsIHNwb3RzOiBbTU9CX3NwYXRpYWxfc3BvdF9sb2MuY3N2XShodHRwczovL2dpdGh1Yi5jb20vYXo3amgyL1NEZVBFUl9BbmFseXNpcy9ibG9iL21haW4vUmVhbERhdGEvTU9CL01PQl9zcGF0aWFsX3Nwb3RfbG9jLmNzdikuIERpcmVjdGx5IGV4dHJhY3QgdGhlIHNwYXRpYWwgYHhgIGFuZCBgeWAgY29vcmRpbmF0ZXMgZnJvbSBzcG90IG5hbWVzLCBmb2xsb3dlZCBieSAqKnJvdW5kaW5nIHRvIG5lYXJlc3QgaW50ZWdlcnMqKi4KICAgICogQWRqYWNlbmN5IE1hdHJpeDogW01PQl9zcGF0aWFsX3Nwb3RfYWRqYWNlbmN5X21hdHJpeC5jc3ZdKGh0dHBzOi8vZ2l0aHViLmNvbS9hejdqaDIvU0RlUEVSX0FuYWx5c2lzL2Jsb2IvbWFpbi9SZWFsRGF0YS9NT0IvTU9CX3NwYXRpYWxfc3BvdF9hZGphY2VuY3lfbWF0cml4LmNzdikuIFNwb3RzIHdpdGhpbiBuZWlnaGJvcmhvb2QgYXJlIGFkamFjZW50ICoqbGVmdCoqLCAqKnJpZ2h0KiosICoqdG9wKiogYW5kICoqYm90dG9tKiogc3BvdHMuCgoKMi4gKipSZWZlcmVuY2Ugc2NSTkEtc2VxIGRhdGEgcHJlcHJvY2Vzc2luZyoqOgoKICAgIDIuMSBJbnB1dCBvcmlnaW5hbCBkYXRhIGZpbGVzCiAgICAKICAgIHNjUk5BLXNlcSBkYXRhIGFyZSBkb3dubG9hZGVkIGZyb20gW0dTRTEyMTg5MV0oaHR0cHM6Ly93d3cubmNiaS5ubG0ubmloLmdvdi9nZW8vcXVlcnkvYWNjLmNnaT9hY2M9R1NFMTIxODkxKS4KICAgIAogICAgKiBSYXcgblVNSSBvZiBhbGwgNTIsNTQ5IHNpbmdsZSBjZWxsczogIFtHU0UxMjE4OTFfT0JfNl9ydW5zLnJhdy5kZ2UuY3N2Lmd6XShodHRwczovL3d3dy5uY2JpLm5sbS5uaWguZ292L2dlby9kb3dubG9hZC8/YWNjPUdTRTEyMTg5MSZmb3JtYXQ9ZmlsZSZmaWxlPUdTRTEyMTg5MSU1Rk9CJTVGNiU1RnJ1bnMlMkVyYXclMkVkZ2UlMkVjc3YlMkVneikgCiAgICAKICAgICogTWV0YSBkYXRhIGZvciAyMSw3NDYgKipOZXVyb25zKiogY2VsbHM6IFtHU0UxMjE4OTFfRmlndXJlXzJfbWV0YWRhdGEudHh0Lmd6XShodHRwczovL3d3dy5uY2JpLm5sbS5uaWguZ292L2dlby9kb3dubG9hZC8/YWNjPUdTRTEyMTg5MSZmb3JtYXQ9ZmlsZSZmaWxlPUdTRTEyMTg5MSU1RkZpZ3VyZSU1RjIlNUZtZXRhZGF0YSUyRXR4dCUyRWd6KQogICAgCiAgICAyLjIgT3V0cHV0IGRhdGEgZmlsZXMgZm9yIGNlbGwgdHlwZSBkZWNvbnZvbHV0aW9uCiAgICAKICAgICogUmF3IG5VTUkgb2YgMTIsODAxIGNlbGxzIHdpdGggc2VsZWN0ZWQgNSBjZWxsIHR5cGVzIGFuZCAxOCw1NjAgZ2VuZXM6IFtNT0JfcmVmX3NjUk5BX2NlbGxfblVNSS5jc3YuZ3pdKGh0dHBzOi8vZ2l0aHViLmNvbS9hejdqaDIvU0RlUEVSX0FuYWx5c2lzL2Jsb2IvbWFpbi9SZWFsRGF0YS9NT0IvTU9CX3JlZl9zY1JOQV9jZWxsX25VTUkuY3N2Lmd6KS4gKipOTyBmaWx0ZXJpbmcgb24gY2VsbHMgb3IgZ2VuZXMqKiwgaS5lLiBhbGwgZ2VuZXMgYW5kIGNlbGxzIHdpdGggdGhvc2UgNSBjZWxsIHR5cGVzIGFyZSBpbmNsdWRlZCBmb3IgYW5hbHlzaXMuCiAgICAKICAgICogQ2VsbCB0eXBlIGFubm90YXRpb24gZm9yIHRob3NlIDEyLDgwMSBjZWxsczogW01PQl9yZWZfc2NSTkFfY2VsbF9jZWxsdHlwZS5jc3ZdKGh0dHBzOi8vZ2l0aHViLmNvbS9hejdqaDIvU0RlUEVSX0FuYWx5c2lzL2Jsb2IvbWFpbi9SZWFsRGF0YS9NT0IvTU9CX3JlZl9zY1JOQV9jZWxsX2NlbGx0eXBlLmNzdikKCgoKIyBWZXJzaW9uCgpgYGB7cn0KdmVyc2lvbltbJ3ZlcnNpb24uc3RyaW5nJ11dCmBgYAoKCiMgUHJlcHJvY2VzcyBNT0Igc3BhdGlhbCBkYXRhc2V0CgojIyBSZWFkIG9yaWdpbmFsIGRhdGEgZmlsZSBbUmVwMTJfTU9CX2NvdW50X21hdHJpeC0xLnRzdl0oaHR0cHM6Ly93d3cuc3BhdGlhbHJlc2VhcmNoLm9yZy93cC1jb250ZW50L3VwbG9hZHMvMjAxNi8wNy9SZXAxMl9NT0JfY291bnRfbWF0cml4LTEudHN2KQoKYGBge3J9CmZpbGVfbmFtZSA9IGZpbGUucGF0aChob21lLmRpciwgJ1JlcDEyX01PQl9jb3VudF9tYXRyaXgtMS50c3YnKQpvcmdfZGF0YSA9IHJlYWQuY3N2KGZpbGVfbmFtZSwgc2VwID0gJ1x0JywgY2hlY2submFtZXMgPSBGLCBoZWFkZXIgPSBULCByb3cubmFtZXMgPSAxKQpwcmludChzcHJpbnRmKCdsb2FkIGRhdGEgZnJvbSAlcycsIGZpbGVfbmFtZSkpCnByaW50KHNwcmludGYoJ3Nwb3RzOiAlZDsgZ2VuZXM6ICVkJywgbnJvdyhvcmdfZGF0YSksIG5jb2wob3JnX2RhdGEpKSkKb3JnX2RhdGFbMTo1LCAxOjVdCmBgYAoKCiMjIFNhdmUgZmlsZXMgZm9yIGRlY29udm9sdXRpb24KCiMjIyBTcGF0aWFsIHNwb3QgblVNSQoKKipObyBmaWx0ZXJpbmcgb24gc3BvdHMgb3IgZ2VuZXMqKiwgZGlyZWN0bHkgc2F2ZSBhbGwgc3BvdHMgYW5kIGdlbmVzIGludG8gZmlsZSBbTU9CX3NwYXRpYWxfc3BvdF9uVU1JLmNzdl0oaHR0cHM6Ly9naXRodWIuY29tL2F6N2poMi9TRGVQRVJfQW5hbHlzaXMvYmxvYi9tYWluL1JlYWxEYXRhL01PQi9NT0Jfc3BhdGlhbF9zcG90X25VTUkuY3N2KS4gKipSb3dzIGFzIHNwYXRpYWwgc3BvdHMgYW5kIGNvbHVtbnMgYXMgZ2VuZXMqKi4KCmBgYHtyfQp3cml0ZS5jc3Yob3JnX2RhdGEsICdNT0Jfc3BhdGlhbF9zcG90X25VTUkuY3N2JykKcHJpbnQoc3ByaW50Zignc2F2ZSAlZCBnZW5lIG5VTUlzIG9mICVkIHNwYXRpYWwgc3BvdHMgaW50byBmaWxlICVzJywgbmNvbChvcmdfZGF0YSksIG5yb3cob3JnX2RhdGEpLCAnTU9CX3NwYXRpYWxfc3BvdF9uVU1JLmNzdicpKQpgYGAKCgojIyMgUGh5c2ljYWwgTG9jYXRpb25zIG9mIHNwYXRpYWwgc3BvdHMKCkRpcmVjdGx5IGV4dHJhY3QgdGhlIHNwYXRpYWwgYHhgIGFuZCBgeWAgY29vcmRpbmF0ZXMgZnJvbSBzcG90IG5hbWVzLCBmb2xsb3dlZCBieSAqKnJvdW5kaW5nIHRvIG5lYXJlc3QgaW50ZWdlcnMqKiwgdGhlbiBzYXZlZCBpbnRvIGZpbGUgW01PQl9zcGF0aWFsX3Nwb3RfbG9jLmNzdl0oaHR0cHM6Ly9naXRodWIuY29tL2F6N2poMi9TRGVQRVJfQW5hbHlzaXMvYmxvYi9tYWluL1JlYWxEYXRhL01PQi9NT0Jfc3BhdGlhbF9zcG90X2xvYy5jc3YpLgoKCmBgYHtyfQpsb2NhbF9kZiA9IGRhdGEuZnJhbWUobmFtZXMgPSByb3cubmFtZXMob3JnX2RhdGEpLCByb3cubmFtZXMgPSByb3cubmFtZXMob3JnX2RhdGEpKQpsb2NhbF9kZiA9IGxvY2FsX2RmICU+JQogIHRpZHlyOjpzZXBhcmF0ZV93aWRlcl9kZWxpbShuYW1lcywgJ3gnLCBuYW1lcyA9IGMoJ3gnLCAneScpKQpsb2NhbF9kZiA9IGFzLmRhdGEuZnJhbWUobG9jYWxfZGYpCnJvdy5uYW1lcyhsb2NhbF9kZikgPSByb3cubmFtZXMob3JnX2RhdGEpCgpsb2NhbF9kZlsneCddID0gcm91bmQoYXMubnVtZXJpYyhsb2NhbF9kZiR4KSkKbG9jYWxfZGZbJ3knXSA9IHJvdW5kKGFzLm51bWVyaWMobG9jYWxfZGYkeSkpCgpsb2NhbF9kZlsxOjUsIF0KCndyaXRlLmNzdihsb2NhbF9kZiwgJ01PQl9zcGF0aWFsX3Nwb3RfbG9jLmNzdicpCnByaW50KHNwcmludGYoJ3NhdmUgUGh5c2ljYWwgTG9jYXRpb25zIG9mIHNwYXRpYWwgc3BvdHMgaW50byBmaWxlICVzJywgJ01PQl9zcGF0aWFsX3Nwb3RfbG9jLmNzdicpKQpgYGAKCgojIyMgQWRqYWNlbmN5IE1hdHJpeCBvZiBzcGF0aWFsIHNwb3RzCgpXZSBkZWZpbmUgdGhlIG5laWdoYm9yaG9vZCBvZiBhIHNwYXRpYWwgc3BvdCBjb250YWlucyB0aGUgYWRqYWNlbnQgKipsZWZ0KiosICoqcmlnaHQqKiwgKip0b3AqKiBhbmQgKipib3R0b20qKiBzcG90LCB0aGF0IGlzLCBvbmUgc3BvdCBoYXMgYXQgbW9zdCA0IG5laWdoYm9ycy4KClRoZSBnZW5lcmF0ZWQgQWRqYWNlbmN5IE1hdHJpeCBgQWAgb25seSBjb250YWlucyAqKjEqKiBhbmQgKiowKiosIHdoZXJlIDEgcmVwcmVzZW50cyBjb3JyZXNwb25kaW5nIHR3byBzcG90cyBhcmUgYWRqYWNlbnQgc3BvdHMgYWNjb3JkaW5nIHRvIHRoZSBkZWZpbml0aW9uIG9mIG5laWdoYm9yaG9vZCwgd2hpbGUgdmFsdWUgMCBmb3Igbm9uLWFkamFjZW50IHNwb3RzLiBOb3RlICoqYWxsIGRpYWdvbmFsIGVudHJpZXMgYXJlIDBzKiouCgpBZGphY2VuY3kgTWF0cml4IGFyZSBzYXZlZCBpbnRvIGZpbGUgW01PQl9zcGF0aWFsX3Nwb3RfYWRqYWNlbmN5X21hdHJpeC5jc3ZdKGh0dHBzOi8vZ2l0aHViLmNvbS9hejdqaDIvU0RlUEVSX0FuYWx5c2lzL2Jsb2IvbWFpbi9SZWFsRGF0YS9NT0IvTU9CX3NwYXRpYWxfc3BvdF9hZGphY2VuY3lfbWF0cml4LmNzdikuCgpgYGB7cn0KZ2V0TmVpZ2hib3VyID0gZnVuY3Rpb24oYXJyYXlfcm93LCBhcnJheV9jb2wpIHsKICAjIGJhc2VkIG9uIHRoZSAocm93LCBjb2wpIG9mIG9uZSBzcG90LCByZXR1cm4gdGhlIChyb3csIGNvbCkgb2YgYWxsIDQgbmVpZ2hib3VycwogIHJldHVybihsaXN0KGMoYXJyYXlfcm93LTEsIGFycmF5X2NvbCksCiAgICAgICAgICAgICAgYyhhcnJheV9yb3crMSwgYXJyYXlfY29sKSwKICAgICAgICAgICAgICBjKGFycmF5X3JvdyswLCBhcnJheV9jb2wtMSksCiAgICAgICAgICAgICAgYyhhcnJheV9yb3crMCwgYXJyYXlfY29sKzEpKSkKfQoKIyBhZGphY2VuY3kgbWF0cml4CkEgPSBtYXRyaXgoMCwgbnJvdyA9IG5yb3cobG9jYWxfZGYpLCBuY29sID0gbnJvdyhsb2NhbF9kZikpCnJvdy5uYW1lcyhBKSA9IHJvd25hbWVzKGxvY2FsX2RmKQpjb2xuYW1lcyhBKSA9IHJvd25hbWVzKGxvY2FsX2RmKQpmb3IgKGkgaW4gMTpucm93KGxvY2FsX2RmKSkgewogIGJhcmNvZGUgPSByb3duYW1lcyhsb2NhbF9kZilbaV0KICBhcnJheV9yb3cgPSBsb2NhbF9kZltpLCAneSddCiAgYXJyYXlfY29sID0gbG9jYWxfZGZbaSwgJ3gnXQogIAogICMgZ2V0IG5laWdoYm9ycwogIG5laWdoYm91cnMgPSBnZXROZWlnaGJvdXIoYXJyYXlfcm93LCBhcnJheV9jb2wpCiAgCiAgIyBmaWxsIHRoZSBhZGphY2VuY3kgbWF0cml4CiAgZm9yICh0aGlzLnZlYyBpbiBuZWlnaGJvdXJzKSB7CiAgICB0bXAucCA9IHJvd25hbWVzKGxvY2FsX2RmW2xvY2FsX2RmJHk9PXRoaXMudmVjWzFdICYgbG9jYWxfZGYkeD09dGhpcy52ZWNbMl0sIF0pCiAgICAKICAgIGlmIChsZW5ndGgodG1wLnApID49IDEpIHsKICAgICAgIyB0YXJnZXQgc3BvdHMgaGF2ZSBuZWlnaGJvcnMgaW4gc2VsZWN0ZWQgc3BvdHMKICAgICAgZm9yIChuZWlnaC5iYXJjb2RlIGluIHRtcC5wKSB7CiAgICAgICAgQVtiYXJjb2RlLCBuZWlnaC5iYXJjb2RlXSA9IDEKICAgICAgfQogICAgfQogIH0KfQoKQVsxOjUsIDE6NV0Kd3JpdGUuY3N2KEEsICdNT0Jfc3BhdGlhbF9zcG90X2FkamFjZW5jeV9tYXRyaXguY3N2JykKcHJpbnQoc3ByaW50Zignc2F2ZSBBZGphY2VuY3kgTWF0cml4IG9mIHNwYXRpYWwgc3BvdHMgaW50byBmaWxlICVzJywgJ01PQl9zcGF0aWFsX3Nwb3RfYWRqYWNlbmN5X21hdHJpeC5jc3YnKSkKYGBgCgpQbG90IEFkamFjZW5jeSBNYXRyaXguIEVhY2ggbm9kZSBpcyBzcG90LCBzcG90cyB3aXRoaW4gbmVpZ2hib3Job29kIGFyZSBjb25uZWN0ZWQgd2l0aCBlZGdlcy4KCmBgYHtyLCBmaWcud2lkdGg9MTIsIGZpZy5oZWlnaHQ9MTJ9CmcgPSBncmFwaF9mcm9tX2FkamFjZW5jeV9tYXRyaXgoQSwgJ3VuZGlyZWN0ZWQnLCBhZGQuY29sbmFtZXMgPSBOQSwgYWRkLnJvd25hbWVzID0gTkEpCiMgbWFudWFsbHkgc2V0IG5vZGVzIHggYW5kIHkgY29vcmRpbmF0ZXMKdmVydGV4X2F0dHIoZywgbmFtZSA9ICd4JykgPSBsb2NhbF9kZiR4CnZlcnRleF9hdHRyKGcsIG5hbWUgPSAneScpID0gbG9jYWxfZGYkeQpwbG90KGcsIHZlcnRleC5zaXplPTUsIGVkZ2Uud2lkdGg9NCwgbWFyZ2luPS0wLjA1KQpgYGAKCgojIFByb3Byb2Nlc3MgcmVmZXJlbmNlIHNjUk5BLXNlcSBkYXRhCgojIyBSZWFkIGFuZCBwcmVwcm9jZXNzIHNjUk5BLXNlcSBtZXRhIGRhdGEKCk9yaWdpbmFsIG1ldGEgZGF0YSBmaWxlIGlzIFtHU0UxMjE4OTFfRmlndXJlXzJfbWV0YWRhdGEudHh0Lmd6XShodHRwczovL3d3dy5uY2JpLm5sbS5uaWguZ292L2dlby9kb3dubG9hZC8/YWNjPUdTRTEyMTg5MSZmb3JtYXQ9ZmlsZSZmaWxlPUdTRTEyMTg5MSU1RkZpZ3VyZSU1RjIlNUZtZXRhZGF0YSUyRXR4dCUyRWd6KSBkb3dubG9hZGVkIGZyb20gW0dTRTEyMTg5MV0oaHR0cHM6Ly93d3cubmNiaS5ubG0ubmloLmdvdi9nZW8vcXVlcnkvYWNjLmNnaT9hY2M9R1NFMTIxODkxKS4gSXQgY29udGFpbnMgbWV0YSBkYXRhIG9mIDIxLDc0NiAqKk5ldXJvbnMqKiBjZWxscyBmcm9tIG1vdXNlIG9sZmFjdG9yeSBidWxiLiBUaGUgY2VsbCB0eXBlIGFubm90YXRpb24gaXMgc3RvcmVkIGluIGNvbHVtbiBgRmluYWxJZHNgLCB3aGljaCBpbmNsdWRlcyB0b3RhbCAxOCBkaXN0aW5jdCBhbm5vdGF0aW9ucy4gV2Ugc2VsZWN0ZWQgNSBjZWxsIHR5cGVzIGFuZCBjb21iaW5lIHRoZSBzdWJ0eXBlcyBhcyBiZWxvdzoKCjEuIGdyYW51bGUgY2VsbHMgKCoqR0MqKik6ICJuMDMtR0MtMSIgKyAibjA3LUdDLTIiICsgIm4wOS1HQy0zIiArICJuMTAtR0MtNCIgKyAibjExLUdDLTUiICsgIm4xMi1HQy02IiArICJuMTQtR0MtNyIKMi4gb2xmYWN0b3J5IHNlbnNvcnkgbmV1cm9ucyAoKipPU05zKiopOiAibjAxLU9TTnMiCjMuIHBlcmlnbG9tZXJ1bGFyIGNlbGxzICgqKlBHQyoqKTogIm4wMi1QR0MtMSIgKyAibjA1LVBHQy0yIiArICJuMDgtUEdDLTMiCjQuIG1pdHJhbCBhbmQgdHVmdGVkIGNlbGxzICgqKk0vVEMqKik6ICJuMTUtTS9UQy0xIiArICJuMTYtTS9UQy0yIiArICJuMTctTS9UQy0zIgo1LiBleHRlcm5hbCBwbGV4aWZvcm0gbGF5ZXIgaW50ZXJuZXVyb25zICgqKkVQTC1JTioqKTogIm4xOC1FUEwtSU4iCgozIFN1YnR5cGVzICJuMDQtSW1tYXR1cmUiLCAibjA2LVRyYW5zaXRpb24iIGFuZCAibjEzLUFzdHJvY3l0ZUxpa2UiIGFyZSBkaXNjYXJkZWQuCgoqKk5PIGZ1cnRoZXIgZmlsdGVyaW5nIG9uIGNlbGxzKiosIGkuZS4gYWxsIDEyLDgwMSBjZWxscyBvZiB0aGVzZSA1IHNlbGVjdGVkIGNlbGwgdHlwZXMgd2lsbCBiZSB1c2VkIGZvciBjZWxsIHR5cGUgZGVjb252b2x1dGlvbi4KCmBgYHtyfQpmaWxlX25hbWUgPSBmaWxlLnBhdGgoaG9tZS5kaXIsICdHU0UxMjE4OTFfRmlndXJlXzJfbWV0YWRhdGEudHh0Lmd6JykKcmVmX21ldGEgPSByZWFkLmNzdihnemZpbGUoZmlsZV9uYW1lKSwgc2VwPSdcdCcsIGNoZWNrLm5hbWVzID0gRiwgaGVhZGVyID0gVCwgcm93Lm5hbWVzID0gMSkKcHJpbnQoc3ByaW50ZignbG9hZCBkYXRhIGZyb20gJXMnLCBmaWxlX25hbWUpKQpwcmludChzcHJpbnRmKCd0b3RhbCAlZCBjZWxscyB3aXRoIGRpc3RpbmN0ICVkIGNlbGwgdHlwZSBhbm5vdGF0aW9ucycsIG5yb3cocmVmX21ldGEpLCBsZW5ndGgodW5pcXVlKHJlZl9tZXRhJEZpbmFsSWRzKSkpKQoKIyByZW1vdmUgdW53YW50ZWQgMyBzdWJ0eXBlcwpyZWZfbWV0YSA9IHJlZl9tZXRhW3JlZl9tZXRhJEZpbmFsSWRzICVub3RpbiUgYygibjA0LUltbWF0dXJlIiwgIm4wNi1UcmFuc2l0aW9uIiwgIm4xMy1Bc3Ryb2N5dGVMaWtlIiksIF0KcHJpbnQoc3ByaW50ZigncmVtb3ZlIDMgY2VsbCBzdWJ0eXBlIGFubm90YXRpb25zLCByZW1haW4gJWQgY2VsbHMnLCBucm93KHJlZl9tZXRhKSkpCgojIGNvbWJpbmUgc3VidHlwZXMKcmVmX21ldGEkY2VsbHR5cGUgPSAiIgpyZWZfbWV0YVtyZWZfbWV0YSRGaW5hbElkcyAlaW4lIGMoIm4wMy1HQy0xIiwgIm4wNy1HQy0yIiwgIm4wOS1HQy0zIiwgIm4xMC1HQy00IiwgIm4xMS1HQy01IiwgIm4xMi1HQy02IiwgIm4xNC1HQy03IiksICJjZWxsdHlwZSJdID0gIkdDIgpyZWZfbWV0YVtyZWZfbWV0YSRGaW5hbElkcyA9PSAibjAxLU9TTnMiLCAiY2VsbHR5cGUiXSA9ICJPU05zIgpyZWZfbWV0YVtyZWZfbWV0YSRGaW5hbElkcyAlaW4lIGMoIm4wMi1QR0MtMSIsICJuMDUtUEdDLTIiLCAibjA4LVBHQy0zIiksICJjZWxsdHlwZSJdID0gIlBHQyIKcmVmX21ldGFbcmVmX21ldGEkRmluYWxJZHMgJWluJSBjKCJuMTUtTS9UQy0xIiwgIm4xNi1NL1RDLTIiLCAibjE3LU0vVEMtMyIpLCAiY2VsbHR5cGUiXSA9ICJNL1RDIgpyZWZfbWV0YVtyZWZfbWV0YSRGaW5hbElkcyA9PSAibjE4LUVQTC1JTiIsICJjZWxsdHlwZSJdID0gIkVQTC1JTiIKCnRhYmxlKHJlZl9tZXRhJGNlbGx0eXBlKQoKcmVmX21ldGFbMTo1LCBjKCdGaW5hbElkcycsICdjZWxsdHlwZScpXQpgYGAKCgpTYXZlIGNlbGwgdHlwZSBhbm5vdGF0aW9uIHRvIGZpbGUgW01PQl9yZWZfc2NSTkFfY2VsbF9jZWxsdHlwZS5jc3ZdKGh0dHBzOi8vZ2l0aHViLmNvbS9hejdqaDIvU0RlUEVSX0FuYWx5c2lzL2Jsb2IvbWFpbi9SZWFsRGF0YS9NT0IvTU9CX3JlZl9zY1JOQV9jZWxsX2NlbGx0eXBlLmNzdikKCmBgYHtyfQp3cml0ZS5jc3YocmVmX21ldGFbLCAnY2VsbHR5cGUnLCBkcm9wPUZdLCAnTU9CX3JlZl9zY1JOQV9jZWxsX2NlbGx0eXBlLmNzdicpCnByaW50KHNwcmludGYoJ3NhdmUgY2VsbCB0eXBlIGFubm90YXRpb24gb2YgcmVmZXJlbmNlIHNjUk5BLXNlcSBjZWxscyBpbnRvIGZpbGUgJXMnLCAnTU9CX3JlZl9zY1JOQV9jZWxsX2NlbGx0eXBlLmNzdicpKQpgYGAKCgojIyBSZWFkIGFuZCBwcmVwcm9jZXNzIHNjUk5BLXNlcSBuVU1JIGRhdGEKCk9yaWdpbmFsIGdlbmUgblVNSSBjb3VudCBkYXRhIGZpbGUgaXMgW0dTRTEyMTg5MV9PQl82X3J1bnMucmF3LmRnZS5jc3YuZ3pdKGh0dHBzOi8vd3d3Lm5jYmkubmxtLm5paC5nb3YvZ2VvL2Rvd25sb2FkLz9hY2M9R1NFMTIxODkxJmZvcm1hdD1maWxlJmZpbGU9R1NFMTIxODkxJTVGT0IlNUY2JTVGcnVucyUyRXJhdyUyRWRnZSUyRWNzdiUyRWd6KSBkb3dubG9hZGVkIGZyb20gW0dTRTEyMTg5MV0oaHR0cHM6Ly93d3cubmNiaS5ubG0ubmloLmdvdi9nZW8vcXVlcnkvYWNjLmNnaT9hY2M9R1NFMTIxODkxKS4gSXQgY29udGFpbnMgdG90YWwgNTIsNTQ5IGNlbGxzIGFuZCAxOCw1NjAgZ2VuZXMuCgpXZSBqdXN0IHNlbGVjdGVkIDEyLDgwMSBjZWxscyBvZiB0aGUgNSBzZWxlY3RlZCBjZWxsIHR5cGVzIGJ5IGJhcmNvZGVzLCBhbmQgZGlzY2FyZCBvdGhlciBjZWxscy4gKipOTyBmaWx0ZXJpbmcgb24gZ2VuZXMqKiwgaS5lLiBhbGwgMTgsNTYwIGdlbmVzIHdpbGwgYmUgdXNlZCBmb3IgY2VsbCB0eXBlIGRlY29udm9sdXRpb24uCgpgYGB7cn0KZmlsZV9uYW1lID0gZmlsZS5wYXRoKGhvbWUuZGlyLCAnR1NFMTIxODkxX09CXzZfcnVucy5yYXcuZGdlLmNzdi5neicpCnJlZl9kYXRhID0gZGF0YS50YWJsZTo6ZnJlYWQoZmlsZV9uYW1lLCBzZXAgPSAiLCIsIGNoZWNrLm5hbWVzID0gRkFMU0UsIHNlbGVjdCA9IGMoJ1YxJywgcm93Lm5hbWVzKHJlZl9tZXRhKSkpCmdlbmVfbmFtZXMgPSByZWZfZGF0YSRWMQoKIyB0cmFuc3Bvc2UgaXQKcmVmX2RhdGEgPSBhcy5kYXRhLmZyYW1lKGRhdGEudGFibGU6OnRyYW5zcG9zZShyZWZfZGF0YSAlPiUKICBzZWxlY3Qocm93Lm5hbWVzKHJlZl9tZXRhKSkpKQoKcm93Lm5hbWVzKHJlZl9kYXRhKSA9IHJvdy5uYW1lcyhyZWZfbWV0YSkKY29sbmFtZXMocmVmX2RhdGEpID0gZ2VuZV9uYW1lcwoKcHJpbnQoc3ByaW50ZignbG9hZCBkYXRhIGZyb20gJXMnLCBmaWxlX25hbWUpKQpwcmludChzcHJpbnRmKCdjZWxsczogJWQ7IGdlbmVzOiAlZCcsIG5yb3cocmVmX2RhdGEpLCBuY29sKHJlZl9kYXRhKSkpCnJlZl9kYXRhWzE6NSwgMTo1XQpgYGAKClNhdmUgc2NSTkEtc2VxIG5VTUkgbWF0cml4IHRvIGZpbGUgW01PQl9yZWZfc2NSTkFfY2VsbF9uVU1JLmNzdi5nel0oaHR0cHM6Ly9naXRodWIuY29tL2F6N2poMi9TRGVQRVJfQW5hbHlzaXMvYmxvYi9tYWluL1JlYWxEYXRhL01PQi9NT0JfcmVmX3NjUk5BX2NlbGxfblVNSS5jc3YuZ3opCgpgYGB7cn0KZGF0YS50YWJsZTo6ZndyaXRlKHJlZl9kYXRhLCAnTU9CX3JlZl9zY1JOQV9jZWxsX25VTUkuY3N2Lmd6Jywgcm93Lm5hbWVzID0gVCkKcHJpbnQoc3ByaW50Zignc2F2ZSBuVU1JIG1hdHJpeCBvZiByZWZlcmVuY2Ugc2NSTkEtc2VxIGNlbGxzIGludG8gZ3ppcCBjb21wcmVzc2VkIGZpbGUgJXMnLCAnTU9CX3JlZl9zY1JOQV9jZWxsX25VTUkuY3N2Lmd6JykpCmBgYAoKCg==