Tuesday, March 17, 2020

Syncing a similar code base between multiple client instances

For years one of the companies I worked for has juggled multiple copies of a very similar code base.  The majority of the code was (or could be) identical, but there was just enough of a difference that a single code base could not be used.  An additional complicating factor was that the company was known by its clients for the ability to quickly turn around feature change requests; which meant that when one client wanted a change, there was not time to test its impact for all clients before rolling it out.

As you can imagine, this was a very difficult process to manage, and very time consuming re-creating identical features when other clients decided they wanted something that had been developed.

Over the years multiple attempts have been made to solve this issue.
- Branching was scrapped because changes can really only be pushed from the base branch to the child/client branches and there was not generally time to test all feature updates when a client wanted just one single change pushed into their code base.
- A shared service architecture was scrapped because versioning quickly became unwieldy between the clients using it, and the shared services started to become fractured.  They also suffered from the same inability to easily test and regression test all combinations of the endpoints.  Also a shared database became a security concern.
- A shared dll was scrapped for similar reasons when one client updated the dll, and the other clients were forced to take all the updates on their next modification.
- Splitting the code into multiple projects by major feature area.  This allowed for smaller pushes when changes were made, and attempts could be made to keep the most similar projects in sync, but it was still unwieldy.

As the features became more numerous regression testing became a big issue.  So a fairly comprehensive automated unit test and UI testing system was developed.  This significantly reduced the danger of moving features between the client code bases, but it did nothing to reduce the time involved.

A lot of posts were reviewed, and a lot of tools tried in an effort to figure out how to have a human easily view and push changes around between all the code bases.

- SyncBackPro (great sync tool, but no human review during the process)
- Winmerge
- Vim
- Diffuse (amazing tool, but crashes on windows with three or more files open)
- Code Compare (best code compare tool found, but only supported up to three files)

Over time Code Compare became the hands down favorite in the company for comparing code files, it was a smoother compare process for code files than any other tool tried; and made pushing changes around much easier.

However, even though it was smoother, it was still a massively huge process; and getting bigger as more and more features were added.  Others have had similar problems, but no one had any amazing and workable general solutions; although many people had tried playing with various git branching type features.  One company even developed a piece of software attempting to tackle this problem, however it appears to be primarily for UI development rather than backend code.

In this ever evolving situation the next attempt build on the popular Code Compare.  A custom powershell script was developed that mapped code bases to either other, or back to a master allowing the code to be instantly compared using Code Compare between a "master" instance of the code so the developer could push new features back to master, and pull down any desired feature differences.

The custom script:
- installs/removes itself in the right click menus for files and folders
- includes directions for installing itself in those menus inside of Visual Studio
- requires the code bases being compared to have identical directory and file structures
- requires that you go through and modify the $parentProjects variable to map your project folders
- uses the $parentProjects mapping to detect the incoming file/folder path, and open a comparison with Code Compare to the file/folder of the corresponding mapped path
Here is the Custom Power Shell Script

#
# VISUAL STUDIO INSTALLATION INSTRUCTIONS:
# Menu: Tools / External Tools
# - Click Add
# - Title: Mas Code Compare
# - Command: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
# - Arguments: -File "D:\Documents\scripts\CompareProjects.ps1" "$(ItemPath)"
# - memorize the index number of the item you just created (how far down in the list it is)
# Menu: Tools / Customize
# - Click the Commands tab
# - choose Context menu: Project and Solution Context Menus | Item
# - - Add Command...
# - - Choose the category Tools
# - - Select External Command
# - choose Context menu: Project and Solution Context Menus | Folder
# - - Add Command...
# - - Choose the category Tools
# - - Select External Command
#





if ($args[0] -eq $null) {
msg *, "Run with the following parameters: -install, -remove, 'PathOfFileOrFolder'"
return
}

#
# Install or Remove the windows context menu item
#
if ($args[0] -eq "-install" -or $args[0] -eq "-remove")
{
# AllFilesystemObjects is the key folder here, it specifies that the "shell" sub folder will be applied
# to all file system objects.
# the "shell" sub folder indicates that we are dealing with the right click context menu
# and the final folder name becomes the name of the menu item itself
$registryPath = "HKCR:\AllFilesystemObjects\shell\MasCodeCompare"
$regAutoPath = "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\CommandStore\shell\MasCodeCompare.Auto"
$regMasCodePath = "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\CommandStore\shell\MasCodeCompare.MasterCode"
$regCli1CodePath = "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\CommandStore\shell\MasCodeCompare.Client1Code"
$regCli2CodePath = "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\CommandStore\shell\MasCodeCompare.Client2Code"
New-PSDrive -Name HKCR -PSProvider Registry -Root HKEY_CLASSES_ROOT
New-PSDrive -Name HKLM -PSProvider Registry -Root HKEY_LOCAL_MACHINE

if ($args[0] -eq "-install")
{
# attempts to get command windows to not show with (/Q and -windowstyle hidden) don't work
$Name = "(Default)"
$value = "CMD.EXE /Q /C Powershell.exe -windowstyle hidden -File "+$PSScriptRoot.replace("\", "\\")+"\\CompareProjects.ps1 %1"

IF(!(Test-Path $registryPath))
{
New-Item -Path $registryPath -Force | Out-Null
New-Item -Path ($registryPath + "\command") -Force | Out-Null
New-Item -Path $regAutoPath -Force | Out-Null
New-Item -Path ($regAutoPath + "\command") -Force | Out-Null
New-Item -Path $regMasCodePath -Force | Out-Null
New-Item -Path ($regMasCodePath + "\command") -Force | Out-Null
New-Item -Path $regCli1CodePath -Force | Out-Null
New-Item -Path ($regCli1CodePath + "\command") -Force | Out-Null
New-Item -Path $regCli2CodePath -Force | Out-Null
New-Item -Path ($regCli2CodePath + "\command") -Force | Out-Null
}

New-ItemProperty -Path ($registryPath + "\command") -Name $name -Value $value -PropertyType String -Force | Out-Null
New-ItemProperty -Path $registryPath -Name "MUIVerb" -Value "Mas Code Compare" -PropertyType String -Force | Out-Null
New-ItemProperty -Path $registryPath -Name "SubCommands" -Value "MasCodeCompare.Auto;MasCodeCompare.MasterCode;MasCodeCompare.Client1Code;MasCodeCompare.Client2Code;" -PropertyType String -Force | Out-Null
New-ItemProperty -Path $regAutoPath -Name "MUIVerb" -Value "Auto choose" -PropertyType String -Force | Out-Null
New-ItemProperty -Path ($regAutoPath + "\command") -Name $name -Value $value -PropertyType String -Force | Out-Null
New-ItemProperty -Path $regMasCodePath -Name "MUIVerb" -Value "Master Code" -PropertyType String -Force | Out-Null
New-ItemProperty -Path ($regMasCodePath + "\command") -Name $name -Value ($value + " Company\MasterCode") -PropertyType String -Force | Out-Null
New-ItemProperty -Path $regCli1CodePath -Name "MUIVerb" -Value "Client 1" -PropertyType String -Force | Out-Null
New-ItemProperty -Path ($regCli1CodePath + "\command") -Name $name -Value ($value + " Client\Client1Code") -PropertyType String -Force | Out-Null
New-ItemProperty -Path $regCli2CodePath -Name "MUIVerb" -Value "Client 2" -PropertyType String -Force | Out-Null
New-ItemProperty -Path ($regCli2CodePath + "\command") -Name $name -Value ($value + " Client\Client2Code") -PropertyType String -Force | Out-Null
return
}
if ($args[0] -eq "-remove")
{
if (test-path $registryPath) { remove-item $registryPath -Recurse }
if (test-path $regAutoPath) { remove-item $regAutoPath -Recurse }
if (test-path $regMasCodePath) { remove-item $regMasCodePath -Recurse }
if (test-path $regCli1CodePath) { remove-item $regCli1CodePath -Recurse }
if (test-path $regCli2CodePath) { remove-item $regCli2CodePath -Recurse }
return
}
}

#
# If we have gotten to here, then we are probably trying to do a compare
#
$itemPath = $args[0]
$compareProject = $args[1]



$parentProjects = @{
"Client\Client2Code" = "Company\MasterCode";
"Client\Client1Code" = "Company\MasterCode";
"Company\MasterCode" = "Client\Client1Code"
}
$parentProject = $null

# find mapping that applies
foreach ($proj in $parentProjects.GetEnumerator())
{
if ($itemPath -like "*" + $proj.Name + "*") {
$parentProject = $proj
break
}
}

# if no parent found, alert user
if ($parentProject -eq $null)
{
Msg * "Invalid project selection"
return
}

# if a parent wasn't requested, then find parent from mapping
if ($compareProject -eq $null)
{
$compareProject = $parentProject.Value
}

$parentPath = $itemPath.replace($parentProject.Name, $compareProject)

& "C:\\Program Files\\Devart\\Code Compare\\CodeCompare.exe" "/environment=auto" "$parentPath" "$itemPath"