Open main menu

CDOT Wiki β

Changes

Code Indexer

148 bytes added, 14:19, 16 January 2007
Status
==Project Contributor(s)==
==Project News and Details======Status====* I got OpenGrok to server, but unfortunatley it gives me a read error permission, so I am re-do ing the installation. --[[User:John64|John64]] 20:15, 23 October 2006 (EDT)* Due to a huge number of assignments I have, this project has stalled until at least reading week. I should now have lots of some time during or soon after reading week to spend resume work on this! --[[User:John64|John64]] 10:02, 19 October 2006 (EDT)*'''Update:''' Since OpenGrok doesn't do file version tracking, There is no point in indexing the entire Mozilla code baseproject. This time I will be pulling down a copy of the Mozilla source tree to index instead. This shouldn't have the intense footprint as an rsync'd Mozilla tree --[[User:John64|John64]] 00:01, 10 October 2006 (EDT)manage my time more wisely* '''Issue:''' OpenGrok is very demanding. It has allready filled a 12GB VMware image, and i still dont know how large the index grows. This could be a problem, not because i dont <strike>I have space, but because this may be too much for the school servermany assignments. Likely OpenGrok would I need a dedicated, reasonibly fast machine. It seems to be a very powerfuldo work on them, extremely well done application but it is missing the ability to associate "blame" from what I see. I also don't know how it takes care of file versioning. This happened when indexing should have more free time during the entire 2.9GB mozilla CVSROOTChristmas break. If you take a look at the example OpenGrok I link to, you will see that its a really nice application!* Switched to a local VM from my P4 --[[User:John64|John64]] 21:44, 4 October 2006 (EDT)* Set Java's location in /etc/default</tomcat5 --[[User:John64|John64]] 20:15, 3 October 2006 (EDT)* Figured out port for Tomcat5, 8180 private. Setting up port forwarding as we speak using public port 81 Oct 3, 2006* Using personal Machine instead of the VM due to it being double NAT-ed, and thus inaccessible from outside the host machine Oct 2, 2006* Set up server using Ubuntu 6.06LTS with Linux 2.6.17ck1 Oct 2, 2006* RSyncing machine Oct 2, 2006strike>
====Candidates==Options==* Help with [http://lxr.mozilla.org/ LXR]/[http://landfill.mozilla.org/mxr-test/ MXR]/[http://www.mozilla.org/bonsai.html Bonsai] - Not working on setting one up because there is already onedevelopment.* Make a sort-of branch respectful version of [http://gonzui.sourceforge.net Gonzui[OpenGrok]] - Impressive looking thing. This will but this would be my first target to setup due to its simplicity and apparently a very powerful natureshoddy implementation that doesn't really do what it should* Setup one OpenGrok per active branch of the Mozilla Project, this would have no version history whatsoever, apart from file dates.* [Re]write major portions of how [[OpenGrok]] - As deals with history and changesets and the likes, this is becoming my main focus, I am breaking it out into its own pagepersonal preference.* Try to fit [http://www.cenqua.com/fisheye/index.html Fisheye] - Commercial Solution that is free (as in beer) into the current development model, but it seems this might be more like finding a problem for open source projectsa solution. Before I start to look at This is a very very powerful tool, but itis not really like LXR or OpenGrok, I would like it is more useful to exhaust analyze CVS/SVN histories more than search for functions, files, definitions and the numerous open source prospectslikes. Fisheye is also '''extremely''' slow.* Hosted by Tigris - I forgot the name Within my lan, it takes a long time to do any queries, but its hosted by "Tigris" and over the internet it looked pretty goodis impossible. If you I don't know why, especially since OpenGrok uses the name, please edit as appropriate* [http://savannahsame basic technologies.nongnu.org/projects/horus Horus] - Not really what is needed(10 minutes plus for one page load on a lan connection, but in its a nice interface for programming students own code. I will not be actively doing anything to it.* [http://bazaar-vcs.org Bazaar] - I dont really know what this isdefence, it might be what is needed, but it might be something irrelevant.* [http://sourceforge.net/projects/sourcenav/ Sourcenav] - investigating this project* [http://google.com/codesearch Google Code Search] - No way to specify still indexing the Mozilla project onlycode)
====LinksWhy I like OpenGrok====Apart from the fact that it does not support branches, this is in my opinion the perfect tool. It is fast, open souce and most importantly, it makes really easy to navigate, well thought out pages that just work. Because of the way OpenSolaris does file versions for their code, they don't use branches at all. OpenSolaris currently uses a linear method of file versioning, they don't use branches, they use versions as a sory of branch, basically the idea that Office 12 is the "2007 branch" and Office 11 is the "2003 branch". Mozilla doesn't do this, so it would be nessecary to implement this feature. Luckily, however, OpenGrok is very modularized and atomic in nature. If you go to the OpenGrok page, you can get a more complete explanation, but the basic jist of it is that there are many "Guru's", each with a task. The files are first read by the History Guru who looks at the file and decides what type of versioning the file uses. Once the versions have been analyzed, they are passed on to the file analyzer guru who then decides what type of file it is, and passes it on to a file type analyzer. The allows for portions of the code to be changed without changing the whole system, so if we wanted to be able to do special things with XUL/XPCOM as far as how to handle its symbols, we would write one module which is not dependent at all on any other file analyzer. The same way, if Mozilla switchs to SVN, we would just port the branching support to SVN. On the chances that Mozilla switches to something other than CVS or SVN, a HistoryGuru could be written for that type of versioning history. The OpenGrok project is under the [http://www.sun.com/cddl/ CDDL] which [http://www.sun.com/cddl/CDDL_MPL_redline.pdf derives from the MPL 1.1] In closing, I really like the OpenGrok project because it is '''very''' fast, '''very''' powerful and '''VERY''' modular! ==Links==
* [https://sparc.senecacollege.ca/portal.php?project&pid=23 Official Blurb] just in case I forget what I am doing :P
* [http://matrix.senecac.on.ca/~jhford/ John's School Page]
* [http://cvs2svn.tigris.org CVS2SVN] Tool to convert CVS to SVN. Will be used to test SVN interop.
* [http://endeveloper.wikipediamozilla.org/wikien/Apache_Tomcat Tomcat on Wikipediadocs/Mozilla_Source_Code_Via_CVS Devmo:CVS Checkout]* [http://developer.mozilla.org/en/docs/Mozilla_Source_Code_Via_CVS Rsyncing_the_CVS_Repository Devmo: Rsyncing the CVS Checkout]* [http://developer.mozilla.org/en/docs/Rsyncing_the_CVS_Repository Rsyncing the CVS_Tags Devmo: CVSTags]- to get the branches to checkout
* [http://www.ubuntuforums.org/showthread.php?t=219985 Tomcat5 on Ubuntu]
* [http://www.onjava.com/pub/a/onjava/2003/06/25/tomcat_tips.html Tomcat Tips]
* [http://atucker.typepad.com/blog/2005/11/a_new_source_br.html Blog Entry] on OpenGrok
* [http://ubuntuforums.org/showthread.php?t=124431 Java5 on Ubuntu] - "sudo update-alternatives --config java" and "apt-get remove --purge java-gcj-compat"
* [http://tomcatwww.apachemozilla.org/tomcat-4.1-docdocs/security-manager-howtojargon.html Tomcat4 DocMozilla Jargon] - possibilities for setting up catalina.policy correctly* [http://developerwww.mozilladeitel.orgcom/enCodeSearchEngines/docs/CVS_Tags CVS TagsCodeSearchEngines_ResourceCenter_MerobaseOpenGrokCodeProject.html Misc] - to get the branches to checkout* [http://tomcatwww.apachesun.orgcom/tomcat-5.5-doccddl/ssi-howtoCDDL_why_details.html Tomcat SSI HowtoCDDL==Notes on Accessing Test Server==If you want to access - explanation of the test server through anything other than port 80, you are going to have to type in diferences between the following address into a browser MPL and note the IP address you get in your address bar. This is because I have dynamic DNS. Everything, including the source itself, will be in the http root for easy access to the files. This is not optimal, and will not stay this way once things advance. [http://superfind.bounceme.net Superfind] - Will resolve as www.no-ip.com computer, which is why you have to use a browser to get the IP ==Questions==Please edit in an answer if you knowCDDL
==Pulling CVS==
This code will pull the CVS for the branches specified in @branches, or it did at some point, your mileage may vary
<pre>
#!/usr/bin/perl
# Pull CVS from the mozilla project server
# Where you want the branch folders
my $src_root = "/var/mozilla";
# Where is your make executable run.sh for opengrok? (Mozilla needs 3.80, nothing elseor equivalent script to start the indexer)my $make opengroker = "/homevar/jhfordopengrok/oldmake/makerun.sh";
# Where is your run.sh for opengrok??server?my $opengroker cvsserver = "/var/opengrok':pserver:anonymous@hera.senecac.on.ca:/run.sh"cvsroot';
# Branches to be pulled
my @branches = (
"HEAD", "REFLOW_20061031_BRANCHMOZILLA_1_8_BRANCH",
"MOZILLA_1_8_0_BRANCH",
"AVIARY_1_0_1_20050124_BRANCH", "REFLOW_20061031_BRANCH" ); # Descriptions for each branch, don't delete old ones for the sake of deleting themmy %descriptions = ( "HEAD" => "Trunk - development branch", "MOZILLA_1_8_BRANCH" => "Firefox 2.0 - development branch", "MOZILLA_1_8_0_BRANCH" => "Firefox 1.5 - maintainance branch", "MOZILLA_1_7_BRANCH" => "Firefox 1.0 - maintainance branch", "AVIARY_1_0_1_20050124_BRANCH" => "Suite - maintainance branch", "REFLOW_20061031_BRANCH" => "Reflow Refactoring"
);
# Open the file or
open BRANCHLIST, ">$branchlistpath" or die "Could not open file";
 
# Clear out what ever source was there
system ("rm -rf ${src_root}/*");
foreach (@branches){
# Download the makefile, then checkout from the makefile
system("
mkdir ${src_root}/$_;
cd ${src_root}/$_;
cvs -d :pserver:anonymous:anonymous\x40cvs-mirror.mozilla.org:/cvsroot ${cvsserver} co -r $_ mozilla/client.mk;
make -f ${src_root}/${_}/mozilla/client.mk checkout MOZ_CO_PROJECT=all;
");
system ("bash $opengroker");
 
</pre>
1
edit