Friday, April 22, 2011

Package the Weave Server - 0.3

Unfortunately the review process for Milestone 0.3 is not complete as I'm still awaiting sponsorship. However some recent changes have occurred, the biggest of which is adapting the package to use the original upstream sources.

Prior to 0.3 the package was using a source tarball made by myself including the sync and reg server components as well as pre-configured conf files. In order to use the two upstream sources sync-server was added as Source0, and reg-server as Source1:
Source0: http://hg.mozilla.org/services/sync-server/archive/bc2f6caf8fc8.tar.bz2 
Source1: http://hg.mozilla.org/services/reg-server/archive/2a73b7376e2c.tar.gz
The %prep section of my SPEC file was changed to extract both sources and strip out unnecessary files (Ex: mercurial meta files).

Note: %{sync} and %{reg} are global macros added to the SPEC to simplify the mercurial tag directory paths.
%prep
%setup -q -n %{sync}
rm -rf .hg* tests/ README 
%setup -q -T -D -n %{reg} -b 1
rm -rf .hg* *.list tests/ README 
cp -p %{SOURCE2} .
In addition two new sources were added to the SPEC for the pre-configured PHP constant files which are copied to their respective component paths in the %install section:
cp -p %{SOURCE3}   %{buildroot}%{_datadir}/%{name}/%{sync}/1.1/
cp -p %{SOURCE4}   %{buildroot}%{_datadir}/%{name}/%{reg}/1.0/
The other significant change to the package is the Apache conf file. It is now commented out by default until the admin decides to begin using Weave. This is done to avoid having Weave active on the web server until the MySQL script has been loaded or if the admin wishes to manually configure the server (Ex: add SSL support). The included README.fedora has been updated to inform the user of this.

I will be continuing to work on getting the package changed and approved past the semester end, to keep up to date with the progress please visit the project wiki and Bugzilla review request.

Sunday, April 10, 2011

Repositories/Distributing Lab

Signing Package
Generated my GPG key using the: gpg --gen-key command. Edited the ~/.rpmmacros file and added the following line:
%_gpg_name "myemail" 
Proceeded to sign the my two packages created and tested in the previous labs (rush and httptunnel).
rpm --addsign httptunnel-3.0.5-1.fc14.rpm
rpm --addsign rush-1.7-1.fc14.rpm
Creating YUM Repo
Created the serve folder as /var/www/html/repo/. Creating the repo metadata with the command:
createrepo /var/www/html/repo/
Copied both of my signed packges to the folder.


Testing
Yum Repo File

Copied /etc/yum.repos.d/fedora.repo as packages.repo, changing it to point to my internal http repo. From my regular user account I used the command:
gpg --export --armour myemail
This generated my GPG key (packages-key) which I copied to the /etc/pki/rpm-gpg. The repo config points to this for signing verification.

After this I was able to issue the command "yum install rush" to get my package. For httptunnel this does not work as the version and release are higher than my current package. If I disable the Fedora repo temporarily it will work.


Repo-release RPM
Spec File
RPM

*Note: The repo-release RPM is pointing to an internal-facing ONLY apache server, the repo will not function on your machine.


In order to figure out how to package my repo configuration I downloaded the SRPM for rpmfusion. Then once issuing the rpm -i command on the source rpm I was able to see the SPEC file it generated.

I adapted the original RPMfusion spec file until it properly installed the packages.repo and packages-key files to /etc/yum.repo.d/ and /etc/pki/rpm-gpg respectively. Installing the RPM allows me to access my internal http repo with no issue.

Once I have an internet-facing server up and running I will find what I've learned from this lab extremely useful.

Mock and Koji Lab

Mock
Build Log - rush
Build Log - httptunnel

I started off by running Mock against my Rush SRPM as it was more complicated to package compared to httptunnel. Used the following command in my SRPM build directory:
mock -r default rush-1.7-1.fc14.src.rpm
The -r default specifies the default fedora release, which was Fedora-14-i386 on my machine. Mock successful built the package without any adjustments needed to my spec's BuildRequires.

Next was httptunnel. The mock command above was repeated with httptunnel's SRPM. Just as with the rush Mock successfully built the package with no changes necessary.


Koji
Task Info - rush
Task Info - httptunnel

Using the command: koji build dist-f14 --scratch SRPM, both packages were submitted to the Koji system for building. Both packages successfully built for i686 and x86_64 architectures.

Reflections
I was pleasantly surprised both of my packages required no modification to build on Mock and Koji. The process was very quick, with both packages not taking more than a few minutes to build with both tools. 


The tools themselves are very handy and powerful. Without Mock for proper build testing a clean machine would be necessary, so obviously the tool is a handy replacement. Koji's ability to build test server-side across multiple architectures (especially ARM) is obviously something that would be different to replicate using your own local machines. Both tools have been a handy assist for my Project milestones. 

RPM-Writing Lab

httptunnel-3.0.5

Source Tar
Spec
SRPM
RPM
Build Log

Created template spec file, failed to build until I added the following macros under %files
%{_bindir}/*
%{_mandir}/*/*

Added first changelog entry for release 1. RPMlint reported issues with formatting and spelling which were corrected until no more lint errors were reported.


rush-1.7

Build Log

Originally the second package for this lab was going to be barcode. However, after experiencing a number of errors during build I decided to look at the original SPEC file for the package. It required a number of patches and additional spec file entries I wouldn't have been able to figure out unless I was knowledgeable about the program. Because of this I decided to choose a different package which I could hopefully build and correct without referring to the original package spec.

Rush required a bit more work and investigation compared to httptunnel which was rather simple to package. The first step was to deal with the install-info files. Upon reading the a number of packaging wikis I determined the SPEC file required the following additions:
Requires(post): info
Requires(preun): info
%post
/sbin/install-info %{_infodir}/%{name}.info %{_infodir}/dir || :
%preun
if [ $1 = 0 ] ; then
/sbin/install-info --delete %{_infodir}/%{name}.info %{_infodir}/dir || :
fi
As well the following additions were required for the %files section in order to handle the locale messages, documentation, and the executable/configuration files. The
%files
%defattr(-,root,root,-)
%doc README
%config %{_sysconfdir}/rush.rc
%config(noreplace) %{_sysconfdir}/rush.rc
%lang(nl) %{_datadir}/locale/nl/LC_MESSAGES/rush.mo
%lang(pl) %{_datadir}/locale/pl/LC_MESSAGES/rush.mo
%lang(uk) %{_datadir}/locale/uk/LC_MESSAGES/rush.mo
%lang(vi) %{_datadir}/locale/vi/LC_MESSAGES/rush.mo
%{_bindir}/*
%{_sbindir}/*
%{_datadir}/rush/rush-po.awk
%{_infodir}/rush.info.gz
After these additions RPMlint reported no errors on the SPEC and SRPM. However, on the RPM itself four errors were reported:

rush.i686: E: non-readable /etc/rush.rc 0640L
The file can't be read by everybody. If this is expected (for security
reasons), contact your rpmlint distributor to get it added to the list of
exceptions for your distro (or add it to your local configuration if you
installed rpmlint from the source tarball).

rush.i686: W: no-manual-page-for-binary rushlast
Each executable in standard binary directories should have a man page.

rush.i686: W: no-manual-page-for-binary rushwho
Each executable in standard binary directories should have a man page.

rush.i686: W: no-manual-page-for-binary rush
Each executable in standard binary directories should have a man page.

1 packages and 0 specfiles checked; 1 errors, 3 warnings.

These errors appear to be issues with the source itself and a lack of proper documentation and possibly permission settings (rush.rc being unreadable).

Monday, April 4, 2011

Package the Weave Server - 0.2

I have managed to successfully package my initial release of the Weave server. The download links for the package can be found at the Package the Weave Server wiki. This wiki will be updated with the latest releases as package review process goes forward. You can also find a link to my Bugzilla submission from the wiki as well.

You can try out weave for yourself by installing the RPM and reading the documentation in the '/usr/share/doc/weave' directory.

Issues Encountered

The main bulk of the work was getting the Weave server to function from a manual installation. The first issue I encountered was figuring out the required dependencies as these are not listed in the setup guides. The one that had me stuck was the driver: 'php-mysql'. Fortunately when I mentioned to Chris Tyler the 'missing driver' statement in my error logs he was able to identify the issue.

In addition, when setting the client I found out it is necessary to include a forward slash in the server address field. Otherwise the client attempts to access http://server1.1/index.php as opposed to the valid url: http://server/1.1/index.php.

For the SPEC file build I did not encounter too many issues. As weave consists of PHP files and does not require any compiling it was simply a matter of copying files to their correct directories, one of the most important being the weave.conf apache config which needed to be copied to /etc/httpd/conf.d to avoid user intervention in the main apache conf.

Unfortunately for the MySQL portion of the Weave install some user interaction is required. The bundled weave.sql script must be loaded to create the necessary database, tables, and service accounts. Additional steps are required if the user wishes to change the default password for the weave dbuser.

Sunday, March 27, 2011

Package the Weave Server - 0.1 (Manual Install)

Hello everyone, it's been a while since my last blog post. Unfortunately I had some health issues to deal with, but things have improved and I will be posting details on my packaging project followed by the labs I've completed.

Milestone 0.2 will deal with the actual packaging of Weave and the bugzilla process, for this post (0.1) I will give you some brief information on what Weave is and how to successfully install and configure it manually.

Package the Weave Server Wiki (For more details)

What is Weave?

Weave is the server component for a product you be aware of called Firefox Sync (now standard with Firefox 4.0). It allows you to backup and sychronize data from your browser or mobile client to Mozilla servers or custom ones. My project is to package Weave for inclusion on the official Fedora repos to assist in the automation and management of custom Weave servers.

Weave consists of two main components: reg-server and sync-server. The first (reg-server) handles account registration and modifications while sync-server handles data collection and management.

Installation:

Before we begin the install a number of packages will be required. Weave requires MySQL and Apache running, PHP including mbstring and mysql driver. We will also be using mercurial to clone the necessary files from Mozilla.
yum install php php-mbstring php-mysql mysql mysql-server mercurial httpd
Next we will clone the reg/sync files from Mozilla.
hg clone http://hg.mozilla.org/services/sync-server
hg clone http://hg.mozilla.org/services/reg-server
This will create directories for both Weave components. Move these to your preferred install directory as your the apache weave conf file will point to it later.

MySQL Setup

Next we will setup the weave database and management user account. I have an SQL script (modified from Anna Sobiepanek's original) which you can use to automate the process. Please download the script from my matrix account.

Note: This script will create the management account: 'weave' with a default password of 'weave', if you wish change the following line in the script with the password of your choice, otherwise leave it default:
CREATE USER 'weave'@'localhost' IDENTIFIED BY 'weave';
Make sure you've setup your root MySQL (or account with permission to create users and database) account and apply the script with the following command:
mysql -u'dbuser' -p'dbpass' < '/script/path/weave.sql'
Configuring Constants

Next we need to configure the PHP constant files for both the sync and reg components. Begin by copying the template .dist files to the same folder without the extension:
cp 'install/path/sync-server/1.1/default_constants.php.dist' '/install/path/sync-server/1.1/default_constants.php'

cp '/install/path/reg-server/1.0/weave_user_constants.php.dist' '/install/path/reg-server/1.0/weave_user_constants.php'
For the purposes of this install we will leave the constants at their default values unless you setup a custom password for the weave MySQL account. Please change the following define statements if you've done so

sync-server/1.1/default_constants.php
if (!defined('WEAVE_MYSQL_STORE_READ_PASS')) { define('WEAVE_MYSQL_STORE_READ_PASS', 'weave'); }
if (!defined('WEAVE_MYSQL_AUTH_PASS')) { define('WEAVE_MYSQL_AUTH_PASS', 'weave'); }
reg-server/1.0/weave_user_constants.php
if (!defined('WEAVE_MYSQL_AUTH_PASS')) { define('WEAVE_MYSQL_AUTH_PASS', 'weave'); }
Apache Config

Create a config file in /etc/httpd/conf.d/ called 'weave.conf' and enter the following aliases pointing to your install directories. The multiple aliases are required for backwards compatibility with older weave clients.
Alias /1.0 <path to install directory>/sync-server/1.1/index.php
Alias /1.1 <path to install directory>/sync-server/1.1/index.php
Alias /user/1.0 <path to install directory>/reg-server/1.0/index.php
Alias /user/1 <path to install directory>/reg-server/1.0/index.php
Make sure to restart the apache service after saving the config file:
service httpd restart
Client Setup (Firefox 4.0 Sync)

Mozilla has a handy Firefox Sync setup video on their website. Point the client to your custom server (http://localhost/) and test your custom weave server by creating an account and syncing.

*Note: Due to a bug with Firefox Sync Client, make sure to include the forward slash / at the end of your server address. Otherwise sync will not work.

Wednesday, January 12, 2011

Build-from-Source Lab

I've chosen the following GNU software packages for this lab: barcode & httptunnel
 

I started the lab by group installing dev tools/libraries as instructed (
yum groupinstall "Development Tools" "Development Libraries").
 

I ran into no issues configuring and compiling the source code for these two GNU applications. The make times are as follows:
 

Barcode: 
real 0m0.555s 
user 0m0.182s 
sys 0m0.179s

HttpTunnel:

real 0m0.592s
user 0m0.182s 

sys 0m0.256s