Difference between revisions of "MDC Infrastructure"

From CDOT Wiki
Jump to: navigation, search
(Project Details)
(December 13, 2006)
 
(18 intermediate revisions by the same user not shown)
Line 3: Line 3:
  
 
== Project Description ==
 
== Project Description ==
Intergrate new features to MDC, work on existing bugs and fix compatibility issues for the upgrade of Mediawiki to 1.7. (Tentative description)
+
Intergrate new features to MDC, work on existing bugs and fix compatibility issues for the upgrade of Mediawiki to 1.7.
  
References: dria and [[User:sancus| sancus]]
+
References: dria and [[User:sancus| sancus]] (primary contact)
  
 
== Project Leader(s) ==
 
== Project Leader(s) ==
Line 11: Line 11:
  
 
== Project Contributor(s) ==
 
== Project Contributor(s) ==
Name(s) of people casually working on the project, or who have contributed significant help. Include links to personal pages within wiki
+
[[User:Eldavey|Erin Davey]] - Provided lots of sample code and implementation details for writing test cases with SimpleTest.
  
 
== Project Details ==
 
== Project Details ==
Line 36: Line 36:
 
</td>
 
</td>
 
<td style="background-color: lime; text-align: center; font-weight: bold; font-size: bigger">DONE</td>
 
<td style="background-color: lime; text-align: center; font-weight: bold; font-size: bigger">DONE</td>
 +
</tr>
 +
<tr>
 +
<td style="text-align: center; font-weight: bold">Nutch-related bugs</td>
 +
<td>Bug: [https://bugzilla.mozilla.org/show_bug.cgi?id=358112 #358112]
 +
* Write code - ''completed 11/15'' ([http://melz.last-chance.net:2080/en/docs/extensions/SpecialNutch.php.diff.txt diff file] | [http://melz.last-chance.net:2080/en/docs/index.php/Special:Nutch result])
 +
* Write test case - [http://melz.last-chance.net:2080/en/docs/tests/nutch_test.php test 1] ([http://melz.last-chance.net:2080/en/docs/tests/nutch_test.phps code]) | [http://melz.last-chance.net:2080/en/docs/tests/Nutch_FormTest.php test 2] ([http://melz.last-chance.net:2080/en/docs/tests/Nutch_FormTest.phps code])
 +
Bug: [https://bugzilla.mozilla.org/show_bug.cgi?id=347924 #347924]
 +
* Feature #1: "Follow the user-set preference for number of results per page"
 +
** Write code
 +
** Write test case
 +
* Feature #2: "Proper Pagination rather than simple Next/Previous links"
 +
** Write code
 +
** Write test case
 +
</td>
 +
<td style="background-color: yellow; text-align: center; font-weight: bold; font-size: bigger">IN PROGRESS</td>
 
</tr>
 
</tr>
 
</table>
 
</table>
 +
I wrote a [[MDC_Infrastructure/Getting_Started|short guide to getting started on this project]] which includes some information on working with [https://bugzilla.mozilla.org/show_bug.cgi?id=347924 bug #347924].
  
Upcoming tasks list (no particular order):
+
Possible future tasks (not planned for this semester - talk to sancus for more details):
 
* Urchin integration
 
* Urchin integration
 
* Work on skin-related bugs
 
* Work on skin-related bugs
Line 51: Line 67:
 
** Problem: After checking out the wiki source from devmo, I used sancus's devmo configuration file to get this running, but it kept redirecting to his installation for some reason. Browsing the setting files, etc. didn't reveal any solution.
 
** Problem: After checking out the wiki source from devmo, I used sancus's devmo configuration file to get this running, but it kept redirecting to his installation for some reason. Browsing the setting files, etc. didn't reveal any solution.
 
** Solution: sancus and shaver suggested using a hostname instead of a user directory link (ie. /~melz). Strangely, that fixed it. A mystery.
 
** Solution: sancus and shaver suggested using a hostname instead of a user directory link (ie. /~melz). Strangely, that fixed it. A mystery.
 
 
==== October 11, 2006 ====
 
==== October 11, 2006 ====
 
* Checked out devmo's css files (it was in a different path) and moved it so that everything loads up properly.
 
* Checked out devmo's css files (it was in a different path) and moved it so that everything loads up properly.
Line 58: Line 73:
 
** Solution: Ended up using just 'mysql' and the errors stopped. 'mysqlimport' seems more picky for some reason.
 
** Solution: Ended up using just 'mysql' and the errors stopped. 'mysqlimport' seems more picky for some reason.
 
* [http://melz.last-chance.net:2080/en/docs/index.php/Main_Page All set up and ready to go.]
 
* [http://melz.last-chance.net:2080/en/docs/index.php/Main_Page All set up and ready to go.]
 +
==== October 25, 2006 ====
 +
* Applied patch for bug [https://bugzilla.mozilla.org/show_bug.cgi?id=352411 #352411]. Didn't work first time, no output from the search engine was displayed. Spoke to sancus about it and he managed to debug the issue so it's working now.
 +
* New bugs to look at: 358112, 357682, 347924 or 353210/353208
 +
* Discussed bug #357682 with sancus and got it narrowed down to [http://wiki.apache.org/nutch/GettingNutchRunningWithUtf8 a server configuration problem] rather than on Nutch's side. (Update: sancus has closed this bug)
 +
* Working on #358112 now...
 +
==== December 13, 2006 ====
 +
Wrapping things up:
 +
* Original test case works correctly again ([http://melz.last-chance.net:2080/en/docs/tests/Nutch_UnitTest.php output link] - takes awhile to finish). My last modification while working on bug #358112 included a strange edit to the SpecialNutch_class.php file which redefined the $language variable (I have no idea how/when I added that line) causing the search engine to break - thus, my failure to demo the actual search engine working from the test site during the presentation. This really stumped me for awhile because I'm not supposed to edit that file in the first place.
 +
* My fix for #358112 is here: [http://melz.last-chance.net:2080/en/docs/extensions/SpecialNutch.php.diff.txt SpecialNutch.php] diff file. What it does: When you access the [http://melz.last-chance.net:2080/en/docs/index.php/Special:Nutch Nutch search form] directly, it should only show a field for you to enter your search term. Since we are using English, the output automatically sets the language to 'en' (English) and no longer displays the drop down box ([http://developer.mozilla.org/en/docs/index.php/Special:Nutch as seen on MDC currently]).
 +
* Thanks to Erin's help, I managed to write a basic test case to test my code. However, it only tests the 'en' language - ([http://melz.last-chance.net:2080/en/docs/tests/nutch_test.php demo] | [http://melz.last-chance.net:2080/en/docs/tests/nutch_test.phps code]). The reason is because the code obtains the language from the URL that calls it ($_SERVER['PHP_SELF']). After reading up on a lot of documention, the only way to test other languages was create install the wiki for every language and test the URL string - which isn't feasible.
 +
* Since there is no direct way that I can test every language, I decided to write a unit test that tests the code I used SpecialNutch.php. This tests constructs a URL for every language, eg. for Korean, the URL will be http://developer.mozilla.org/ko/docs/Special:Nutch. Then the code (being tested) will extra out 'ko' as the language and tests to make sure that is correct ([http://melz.last-chance.net:2080/en/docs/tests/Nutch_FormTest.php demo] | [http://melz.last-chance.net:2080/en/docs/tests/Nutch_FormTest.phps  code]).
 +
* Wrote a [[MDC_Infrastructure/Getting_Started|short guide to getting started on this project]], and avoiding the problems I had.
  
 
== Project Resources ==
 
== Project Resources ==
 
* [http://developer.mozilla.org Mozilla Developer Centre]
 
* [http://developer.mozilla.org Mozilla Developer Centre]
 +
* [http://lucene.apache.org/nutch/ Nutch] - search engine used on MDC. It's written in Java, bleh.

Latest revision as of 20:04, 13 December 2006

Project Name

MDC Infrastructure

Project Description

Intergrate new features to MDC, work on existing bugs and fix compatibility issues for the upgrade of Mediawiki to 1.7.

References: dria and sancus (primary contact)

Project Leader(s)

Melissa Peh

Project Contributor(s)

Erin Davey - Provided lots of sample code and implementation details for writing test cases with SimpleTest.

Project Details

Task Details Status
Mediawiki Setup Setup an installation of devmo wiki (production copy) for local development and testing.
  • Get linux2 account - completed 9/24
  • Checkout devmo mediawiki source from cvs - completed 9/24
  • Get wiki all up and running properly - completed 10/11
  • Setup and move to own database - completed 10/11
DONE
Integrate Nutch searchbox into wiki skin Related bug: #352411
  • Apply patch - completed 10/25
  • Test - completed 10/25
DONE
Nutch-related bugs Bug: #358112

Bug: #347924

  • Feature #1: "Follow the user-set preference for number of results per page"
    • Write code
    • Write test case
  • Feature #2: "Proper Pagination rather than simple Next/Previous links"
    • Write code
    • Write test case
IN PROGRESS

I wrote a short guide to getting started on this project which includes some information on working with bug #347924.

Possible future tasks (not planned for this semester - talk to sancus for more details):

  • Urchin integration
  • Work on skin-related bugs
  • Updating extensions for Mediawiki 1.7 compatibility

Project News

September 20, 2006

  • Discussion of project scope with Deb, Andrei, Dave and Shaver.

September 25, 2006

  • Received an account on linux 2 to install my own local installation of devmo wiki for testing.
    • Problem: After checking out the wiki source from devmo, I used sancus's devmo configuration file to get this running, but it kept redirecting to his installation for some reason. Browsing the setting files, etc. didn't reveal any solution.
    • Solution: sancus and shaver suggested using a hostname instead of a user directory link (ie. /~melz). Strangely, that fixed it. A mystery.

October 11, 2006

  • Checked out devmo's css files (it was in a different path) and moved it so that everything loads up properly.
  • Everything looks great so I'm getting my own dump of devmo's wiki contents instead of pointing to sancus's installation
    • Problem: Had issues getting 'mysqlimport' to populate my database.
    • Solution: Ended up using just 'mysql' and the errors stopped. 'mysqlimport' seems more picky for some reason.
  • All set up and ready to go.

October 25, 2006

  • Applied patch for bug #352411. Didn't work first time, no output from the search engine was displayed. Spoke to sancus about it and he managed to debug the issue so it's working now.
  • New bugs to look at: 358112, 357682, 347924 or 353210/353208
  • Discussed bug #357682 with sancus and got it narrowed down to a server configuration problem rather than on Nutch's side. (Update: sancus has closed this bug)
  • Working on #358112 now...

December 13, 2006

Wrapping things up:

  • Original test case works correctly again (output link - takes awhile to finish). My last modification while working on bug #358112 included a strange edit to the SpecialNutch_class.php file which redefined the $language variable (I have no idea how/when I added that line) causing the search engine to break - thus, my failure to demo the actual search engine working from the test site during the presentation. This really stumped me for awhile because I'm not supposed to edit that file in the first place.
  • My fix for #358112 is here: SpecialNutch.php diff file. What it does: When you access the Nutch search form directly, it should only show a field for you to enter your search term. Since we are using English, the output automatically sets the language to 'en' (English) and no longer displays the drop down box (as seen on MDC currently).
  • Thanks to Erin's help, I managed to write a basic test case to test my code. However, it only tests the 'en' language - (demo | code). The reason is because the code obtains the language from the URL that calls it ($_SERVER['PHP_SELF']). After reading up on a lot of documention, the only way to test other languages was create install the wiki for every language and test the URL string - which isn't feasible.
  • Since there is no direct way that I can test every language, I decided to write a unit test that tests the code I used SpecialNutch.php. This tests constructs a URL for every language, eg. for Korean, the URL will be http://developer.mozilla.org/ko/docs/Special:Nutch. Then the code (being tested) will extra out 'ko' as the language and tests to make sure that is correct (demo | code).
  • Wrote a short guide to getting started on this project, and avoiding the problems I had.

Project Resources