With more and more ace2 addons coming, mostly updated nearly every day the updating of my addons took more and more time because of the externals.
There were some discussions about shaving off externals and just update the core-addon-files but that also is a pain in the ass ;9
Today I wrote a small perl script (you can get perl for windows for freefrom www.activestate.com) that does the following:
1. It reads a file "addons.txt". This files contains the Addons I want.
2. It fetches/upates the addons from SVN, omitting the externals.
3. It reads all svn:external Tags and saves them to a second file
4. It fetches/updates the externals (of course everyone only once even if its used by many addons)
5. it "svn exports" the addons to a seperate folder (so no .svn files there)
6. it exports all externals to the folders within the exported addons.
Please note this is barely tested... Just sharing my stuff for others that might find it useful. If you don't understand what it does, how it does then don't even try it. I tried to make no errors but I don't take responibility for anything.
(of course mine is much longer but I don't want to make this post too long
when I run my script it shows following:
checkout: OneBag
A repos\OneBag\OneBag.toc
A repos\OneBag\version history.txt
A repos\OneBag\BagSlot.blp
A repos\OneBag\BagSlot2.tga
A repos\OneBag\libs
A repos\OneBag\OneBag.xml
A repos\OneBag\OneCore.lua
A repos\OneBag\OneBag.lua
A repos\OneBag\readme.txt
A repos\OneBag\locals.lua
A repos\OneBag\templates.xml
U repos\OneBag
Checked out revision 6109.
checkout: oUF
A repos\oUF\otravi_UnitFramesLocals.lua
A repos\oUF\otravi_UnitFrames.lua
A repos\oUF\textures
A repos\oUF\textures\oUF.tga
A repos\oUF\textures\smooth.tga
A repos\oUF\textures\striped.tga
A repos\oUF\textures\krsnik.tga
A repos\oUF\textures\glaze.tga
A repos\oUF\textures\perl.tga
A repos\oUF\oUF.toc
U repos\oUF
Checked out revision 6109.
checkout: FuBar
A repos\FuBar\FuBar.lua
A repos\FuBar\FuBar.toc
A repos\FuBar\FuBar-Locale-enUS.lua
A repos\FuBar\background.tga
A repos\FuBar\libs
A repos\FuBar\FuBar-Locale-deDE.lua
A repos\FuBar\FuBar_Panel.lua
Checked out revision 6109.
checkout: Squishy
A repos\Squishy\audio
A repos\Squishy\audio\aggro.wav
A repos\Squishy\hooks.lua
A repos\Squishy\textures
A repos\Squishy\textures\smooth.tga
A repos\Squishy\textures\default.tga
A repos\Squishy\Squishy.xml
A repos\Squishy\developer-todo.txt
A repos\Squishy\core.lua
A repos\Squishy\Squishy.toc
A repos\Squishy\libs
A repos\Squishy\credits.txt
A repos\Squishy\Bindings.xml
A repos\Squishy\frames.lua
A repos\Squishy\locals.lua
A repos\Squishy\roster.lua
U repos\Squishy
Checked out revision 6110.
checkout external: AceLibrary
A extern\svn.wowace.com\root\trunk\Ace2\AceLibrary\AceLibrary.lua
Checked out revision 6110.
checkout external: Compost-2.0
A extern\svn.wowace.com\root\trunk\CompostLib\Compost-2.0\Compost-2.0.lua
Checked out revision 6110.
checkout external: AceOO-2.0
A extern\svn.wowace.com\root\trunk\Ace2\AceOO-2.0\AceOO-2.0.lua
Checked out revision 6110.
checkout external: AceAddon-2.0
A extern\svn.wowace.com\root\trunk\Ace2\AceAddon-2.0\AceAddon-2.0.lua
Checked out revision 6110.
checkout external: AceDB-2.0
A extern\svn.wowace.com\root\trunk\Ace2\AceDB-2.0\AceDB-2.0.lua
Checked out revision 6110.
checkout external: AceEvent-2.0
A extern\svn.wowace.com\root\trunk\Ace2\AceEvent-2.0\AceEvent-2.0.lua
Checked out revision 6110.
checkout external: AceHook-2.0
A extern\svn.wowace.com\root\trunk\Ace2\AceHook-2.0\AceHook-2.0.lua
Checked out revision 6110.
checkout external: AceConsole-2.0
A extern\svn.wowace.com\root\trunk\Ace2\AceConsole-2.0\AceConsole-2.0.lua
Checked out revision 6110.
checkout external: AceLocale-2.0
A extern\svn.wowace.com\root\trunk\Ace2\AceLocale-2.0\AceLocale-2.0.lua
Checked out revision 6110.
checkout external: AceModuleCore-2.0
A extern\svn.wowace.com\root\trunk\Ace2\AceModuleCore-2.0\AceModuleCore-2.0.lua
Checked out revision 6110.
checkout external: Dewdrop-2.0
A extern\svn.wowace.com\root\trunk\DewdropLib\Dewdrop-2.0\Dewdrop-2.0.lua
Checked out revision 6110.
checkout external: Jostle-2.0
A extern\svn.wowace.com\root\trunk\JostleLib\Jostle-2.0\Jostle-2.0.lua
Checked out revision 6110.
checkout external: AceDebug
A extern\svn.wowace.com\root\trunk\Ace2\AceDebug-2.0\AceDebug-2.0.lua
Checked out revision 6110.
checkout external: MetrognomeLib
A extern\svn.wowace.com\root\trunk\Metrognome\Lib\MetrognomeLib.lua
Checked out revision 6110.
checkout external: DewdropLib
A extern\svn.wowace.com\root\trunk\DewdropLib\Lib\DewdropLib.lua
Checked out revision 6110.
checkout external: AceTab-2.0
A extern\svn.wowace.com\root\trunk\Ace2\AceTab-2.0\AceTab-2.0.lua
Checked out revision 6110.
checkout external: Metrognome-2.0
A extern\svn.wowace.com\root\trunk\Metrognome\Metrognome-2.0\Metrognome-2.0.lua
Checked out revision 6110.
checkout external: ProximityLib
A extern\svn.wowace.com\root\trunk\ProximityLib\Lib\ProximityLib.lua
Checked out revision 6110.
Exporting: OneBag to Addons\
Exporting: oUF to Addons\
Exporting: FuBar to Addons\
Exporting: Squishy to Addons\
Exporting external: AceLibrary to \FuBar\libs
Exporting external: Compost-2.0 to \FuBar\libs
Exporting external: AceOO-2.0 to \FuBar\libs
Exporting external: AceAddon-2.0 to \FuBar\libs
Exporting external: AceDB-2.0 to \FuBar\libs
Exporting external: AceEvent-2.0 to \FuBar\libs
Exporting external: AceHook-2.0 to \FuBar\libs
Exporting external: AceConsole-2.0 to \FuBar\libs
Exporting external: AceLocale-2.0 to \FuBar\libs
Exporting external: AceModuleCore-2.0 to \FuBar\libs
Exporting external: Dewdrop-2.0 to \FuBar\libs
Exporting external: Jostle-2.0 to \FuBar\libs
Exporting external: AceLibrary to \OneBag\libs
Exporting external: AceOO to \OneBag\libs
Exporting external: AceEvent to \OneBag\libs
Exporting external: AceHook to \OneBag\libs
Exporting external: AceDB to \OneBag\libs
Exporting external: AceDebug to \OneBag\libs
Exporting external: AceLocale to \OneBag\libs
Exporting external: AceConsole to \OneBag\libs
Exporting external: AceAddon to \OneBag\libs
Exporting external: AceModuleCore to \OneBag\libs
Exporting external: DewDrop to \OneBag\libs
Exporting external: MetrognomeLib to \oUF
Exporting external: DewdropLib to \oUF
Exporting external: AceLibrary to \oUF
Exporting external: AceOO to \oUF
Exporting external: AceDB to \oUF
Exporting external: AceConsole to \oUF
Exporting external: AceEvent to \oUF
Exporting external: AceAddon to \oUF
Exporting external: AceHook to \oUF
Exporting external: AceDebug to \oUF
Exporting external: AceAddon-2.0 to \Squishy\libs
Exporting external: AceConsole-2.0 to \Squishy\libs
Exporting external: AceDB-2.0 to \Squishy\libs
Exporting external: AceDebug-2.0 to \Squishy\libs
Exporting external: AceEvent-2.0 to \Squishy\libs
Exporting external: AceHook-2.0 to \Squishy\libs
Exporting external: AceLibrary to \Squishy\libs
Exporting external: AceLocale-2.0 to \Squishy\libs
Exporting external: AceModuleCore-2.0 to \Squishy\libs
Exporting external: AceOO-2.0 to \Squishy\libs
Exporting external: AceTab-2.0 to \Squishy\libs
Exporting external: Dewdrop-2.0 to \Squishy\libs
Exporting external: Metrognome-2.0 to \Squishy\libs
Exporting external: ProximityLib to \Squishy\libs
the script (just put it in an empty folder with the addons.txt file)
Fixed: Damn backslashes don't even work in windows!
use strict;
my $svn = 'C:/Progra~1/Subversion/bin/svn.exe';
die ("$svn not found!") if ( !-e $svn );
mkdir "Addons" if ( !-d "Addons" );
mkdir "repos" if ( !-d "repos" );
mkdir "extern" if ( !-d "extern" );
######## workflow #########
checkout_update_addons();
create_external_list();
checkout_update_externals();
export_addons();
export_externals();
########## subs ###########
sub export_addons
{
open ADDLIST, "addons.txt";
while (<ADDLIST>)
{
my ($aname,$svnpath) = split;
next if (!$aname);
next if (!$svnpath);
my $repos = "repos/".$aname;
print "Exporting: $aname to Addons/\n";
system qq($svn export $repos Addons//$aname --force -q);
}
close ADDLIST;
}
sub export_externals
{
open EXTLIST, "external.cache.txt";
while (<EXTLIST>)
{
my ($target,$extname,$svnpath) = split;
my $extdir = create_external_name($extname,$svnpath);
print "Exporting external: $extname to $target\n";
system qq($svn export $extdir Addons//$target//$extname --force -q);
}
close ADDLIST;
}
sub checkout_update_addons
{
open ADDLIST, "addons.txt";
while (<ADDLIST>)
{
my ($aname,$svnpath) = split;
next if (!$aname);
next if (!$svnpath);
my $repos = "repos//".$aname;
if ( -d $repos )
{
print "update: $aname\n";
system qq($svn up $repos --ignore-externals);
} else {
print "checkout: $aname\n";
system qq($svn co $svnpath $repos --ignore-externals);
}
}
close ADDLIST;
}
sub create_external_name
{
my ($name,$url) = @_;
my ($proto,$dirname);
my @path;
($proto,$url) = split /\/\//,$url,2;
@path = split /\//, $url;
$dirname = "extern/";
foreach (@path)
{
$dirname .= $_."/";
if (!-d $dirname) {
mkdir $dirname or die ("Could not create directory $dirname for $name ($url)");
}
}
return $dirname;
}
sub checkout_update_externals
{
my %updated;
open ADDLIST, "external.cache.txt";
while (<ADDLIST>)
{
my ($dependency,$aname,$svnpath) = split;
my $extname = create_external_name($aname,$svnpath);
next if exists $updated{$extname};
my $repos = "extern/".$aname;
if ( -d $extname."/.svn" )
{
print "update external: $aname\n";
system qq($svn up $extname);
} else {
print "checkout external: $aname\n";
system qq($svn co $svnpath $extname);
}
$updated{$extname} = $aname;
}
close ADDLIST;
}
sub create_external_list_rec
{
my ($dir) = @_;
my $file;
opendir REPOS, "repos//".$dir;
my @dcontent = readdir(REPOS);
closedir (REPOS);
foreach (@dcontent)
{
my $file = $_;
my $dfile = "$dir//$file";
next if (( !-d "repos//".$dfile ) || ($file eq "..") || ($file eq ".") || ($file eq ".svn"));
my $rawext = qx($svn propget svn:externals repos//$dfile --non-interactive --strict);
my @externals = split /\n/,$rawext;
foreach (@externals)
{
print EXTFILE "$dfile $_\n";
}
create_external_list_rec($dfile);
}
}
sub create_external_list
{
open EXTFILE, ">external.cache.txt";
create_external_list_rec("");
close EXTFILE;
}
I had an idea that there could be pipes used as seperator for the externals and I only split for newline... I'm not sure but I'll try to check it here from work ;)
planned: ability to provide a zip file download or curse, wowi, ui.worldofwar, etc url and have it auto-check for updates.
hmm.. you are checking out externals to external/NAME but still trying to export from the old directory (external/path/.../name
I did the structur for externals because there could be externals with the same name from different path in the branches ,9 .. but using File::Path is definitly a good idea :9
With more and more ace2 addons coming, mostly updated nearly every day the updating of my addons took more and more time because of the externals.
There were some discussions about shaving off externals and just update the core-addon-files but that also is a pain in the ass ;9
Today I wrote a small perl script (you can get perl for windows for freefrom www.activestate.com) that does the following:
1. It reads a file "addons.txt". This files contains the Addons I want.
2. It fetches/upates the addons from SVN, omitting the externals.
3. It reads all svn:external Tags and saves them to a second file
4. It fetches/updates the externals (of course everyone only once even if its used by many addons)
5. it "svn exports" the addons to a seperate folder (so no .svn files there)
6. it exports all externals to the folders within the exported addons.
Please note this is barely tested... Just sharing my stuff for others that might find it useful. If you don't understand what it does, how it does then don't even try it. I tried to make no errors but I don't take responibility for anything.
the addons.txt file looks like that:
(of course mine is much longer but I don't want to make this post too long
when I run my script it shows following:
the script (just put it in an empty folder with the addons.txt file)
http://omikrons.de/svn_fetcher.pl
my $svn = 'C:\Program Files\Subversion\bin\svn.exe';
It returns that "c:\Program is not a valid command etc."
my $svn = '"C:\Program Files\Subversion\bin\svn.exe"';
Then it just says it doesn't exist.
And btw it makes 3 folders, Addons is one of them, does that mean i should put it in WoW\Interface or should i copy the addons folder over?
Edit: Stuuuuuupiiid, i was pointing it to "Suversion" not "Subversion", so now it doesn't say it's not found, but during checkout it writes
checkout: BigWigs
'c:\Program' is not recognized as an internal or external command, operable program or batch file.
So some kind of quotation is needed somewhere :)
Didn't work :(
Ok :)
use strict; my $svn = 'C:/Progra~1/Subversion/bin/svn.exe'; die ("$svn not found!") if ( !-e $svn ); mkdir "Addons" if ( !-d "Addons" ); mkdir "repos" if ( !-d "repos" ); mkdir "extern" if ( !-d "extern" ); ######## workflow ######### checkout_update_addons(); create_external_list(); checkout_update_externals(); export_addons(); export_externals(); ########## subs ########### sub export_addons { open ADDLIST, "addons.txt"; while (<ADDLIST>) { my ($aname,$svnpath) = split; next if (!$aname); next if (!$svnpath); my $repos = "repos/".$aname; print "Exporting: $aname to Addons/\n"; system qq($svn export $repos Addons//$aname --force -q); } close ADDLIST; } sub export_externals { open EXTLIST, "external.cache.txt"; while (<EXTLIST>) { my ($target,$extname,$svnpath) = split; my $extdir = create_external_name($extname,$svnpath); print "Exporting external: $extname to $target\n"; system qq($svn export $extdir Addons//$target//$extname --force -q); } close ADDLIST; } sub checkout_update_addons { open ADDLIST, "addons.txt"; while (<ADDLIST>) { my ($aname,$svnpath) = split; next if (!$aname); next if (!$svnpath); my $repos = "repos//".$aname; if ( -d $repos ) { print "update: $aname\n"; system qq($svn up $repos --ignore-externals); } else { print "checkout: $aname\n"; system qq($svn co $svnpath $repos --ignore-externals); } } close ADDLIST; } sub create_external_name { my ($name,$url) = @_; my ($proto,$dirname); my @path; ($proto,$url) = split /\/\//,$url,2; @path = split /\//, $url; $dirname = "extern/"; foreach (@path) { $dirname .= $_."/"; if (!-d $dirname) { mkdir $dirname or die ("Could not create directory $dirname for $name ($url)"); } } return $dirname; } sub checkout_update_externals { my %updated; open ADDLIST, "external.cache.txt"; while (<ADDLIST>) { my ($dependency,$aname,$svnpath) = split; my $extname = create_external_name($aname,$svnpath); next if exists $updated{$extname}; my $repos = "extern/".$aname; if ( -d $extname."/.svn" ) { print "update external: $aname\n"; system qq($svn up $extname); } else { print "checkout external: $aname\n"; system qq($svn co $svnpath $extname); } $updated{$extname} = $aname; } close ADDLIST; } sub create_external_list_rec { my ($dir) = @_; my $file; opendir REPOS, "repos//".$dir; my @dcontent = readdir(REPOS); closedir (REPOS); foreach (@dcontent) { my $file = $_; my $dfile = "$dir//$file"; next if (( !-d "repos//".$dfile ) || ($file eq "..") || ($file eq ".") || ($file eq ".svn")); my $rawext = qx($svn propget svn:externals repos//$dfile --non-interactive --strict); my @externals = split /\n/,$rawext; foreach (@externals) { print EXTFILE "$dfile $_\n"; } create_external_list_rec($dfile); } } sub create_external_list { open EXTFILE, ">external.cache.txt"; create_external_list_rec(""); close EXTFILE; }getting these errors
This how the the external.cache.txt is looking
Seems to be a formatting error, but i have no clue really.
Hope someone can assist :)
using the script posted by neronix.
Edit:
The only addons i currently have this problem with is:
CC_Core
CC_Roll
CC_ToolTip
BulkMail
Mendeleev
I wish i had more brains for this but i did look at it getting even more clueless for about 1½ hour ^^
it seems you're trying to recurse through all directories and get a list of externals, but your $dir and $dfile are not getting set correctly.
http://ohdarn.net/addons/update/update.pl is the version I'm working on. seems to at least work now.
http://ohdarn.net/addons/update/update.diff is a diff of dafire's version.
planned: ability to provide a zip file download or curse, wowi, ui.worldofwar, etc url and have it auto-check for updates.
Another thing I wanted to include is saving the revision number from last update and use
svn log -q --verbose
to get list of changed files and then only update when necessary
but not sure when I can do it because I'm a bit busy at work at the moment.
edit: it was IDCard, none of its externals were exported
hmm.. you are checking out externals to external/NAME but still trying to export from the old directory (external/path/.../name
I did the structur for externals because there could be externals with the same name from different path in the branches ,9 .. but using File::Path is definitly a good idea :9
one addon has added http://svn.wowace.com/root/trunk/PeriodicTable/Elements/ZG as external and later another addon has http://svn.wowace.com/root/trunk/PeriodicTable/Elements/ as external and that's creating headache for the script because the folder already exists ;)
The script itself is super, once working as intended :p and thanks for taking your time to look it thru :)