Rookie python

When running in a virtualenv don’t forget to source bin/activate

Install packages using pip e.g. pip install beautifulsoup4

Create a REQUIREMENTS file using pip freeze and load using pip install -r REQUIREMENTS

Creating a module you need __init__.py in the directory in order to import successfully otherwise you’ll get AttributeError: 'module' object has no attribute 'xxxx'

Javascript debugging using Eclipse

Start Chrome with remote debugging enabled

google-chrome --remote-debugging-port=9222 --user-data-dir=/home/iwright/chrome

Create a new debug configuration using WebKit Protocol

Host localhost
Port 9222
Wip backend Protocol 1.0

Software update site

http://chromedevtools.googlecode.com/svn/update/dev/

grails – the very basics

A lightening fast summary of how to create a CRUD application using grails.
In many ways this is a similar to using Spring Roo (see this post) but with Grails/Groovy instead of Spring Roo.
There are a lot of similarities between the two approaches and, of course, if you want to you can mix java and groovy…

This is a very brief precis of this developer works series

Set up some classes with simple list, edit, delete

From the grails command prompt (Ctrl-Alt-Shift G in STS)

create-domain-class Trip

 

Add some fields (no need for ; )
e.g.

String name
	String city
	Date startDate
	Date endDate
	String purpose
	String notes

 

 

generate-all package.Trip

 

Change content of TripController.groovy to

 

def scaffold = Trip

Remove views (if you don’t want to customize them later) – you can always recreate them with

generate-views Airport

Do the same for Airline

Define many to one relationships

static hasMany = [trip:Trip]

If you want cascading deletes the add a belongsTo (otherwise just declare)

static belongsTo = [Airline]

Set some constraints

	static mapping = {
		table 'some_other_table_name'
		columns {
		  name column:'airline_name'
		  url column:'link'
		  frequentFlyer column:'ff_id'
		}
	  }

	static constraints = {
		name(blank:false, maxSize:100)
		url(url:true)
		frequentFlyer(blank:true)
		notes(maxSize:1500)
	}
	static hasMany = [trip:Trip]
	String name
	String url
	String frequentFlyer
	String notes

	String toString(){
		return name
	}

DB config

install-dependency mysql:mysql-connector-java:5.1.20

In grails-app/conf/BuildConfig.groovy uncomment the mysql dependency

In grails-app/conf/DataSource.groovy

 driverClassName = "com.mysql.jdbc.Driver"
  username = "grails"
  password = "server"
  url = "jdbc:mysql://localhost:3306/trip?autoreconnect=true"

Create a custom taglib

create-tag-lib Date
                
class DateTagLib {
  def thisYear = {
    out << Calendar.getInstance().get(Calendar.YEAR)
  }
}
<div id="copyright">
&copy; 2002 - <g:thisYear />, FakeCo Inc. All Rights Reserved.
</div>
class DateTagLibTests extends GroovyTestCase {
  def dateTagLib

  void setUp(){
    dateTagLib = new DateTagLib()
  }

  void testThisYear() {
    String expected = Calendar.getInstance().get(Calendar.YEAR)
    assertEquals("the years don't match", expected, dateTagLib.thisYear())
  }
}

Fragments

Similar to .jspf – use an _ e.g. _footer.gsp

                
<html><body>
...
<g:render template="/footer" />
</body></html>

Templates

To use your own templates instead of the defaults e.g. for default scaffold

install-templates

Some maven thoughts

Some random observations about using maven

Checking dependencies

mvn versions:display-dependency-updates
mvn versions:display-plugin-updates

Release

With github

Use at least mvn 3.0.4 and release plugin 2.3.2 otherwise it hangs/takes a long time to return apparently after doing git push (it may still work)

When you want to release but don’t want to deploy to a repo

One shot


mvn -o release:prepare -DpreparationGoals='clean install -Dmaven.test.skip=true' -Dresume=false -DautoVersionSubmodules=true

Two shot

mvn release:prepare as normal

mvn release:perform -Dgoals=install

Working with a repository

1) In .m2/settings.xml

 <settings xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd">
<servers>
   <server>
      <id>myrepo-read</id>
      <username>mavenuser</username>
      <password>mavenpass</password>
   </server>

<!-- for deployment -->
 <server>
    <id>myrepo</id>
    <username>user</username>
    <privateKey>/home/user/.ssh/id_rsa</privateKey>
    <passphrase>mypassphrase</passphrase>
    <directoryPermissions>775</directoryPermissions>
    <filePermissions>664</filePermissions>
 </server>
</servers>

<profiles>

    <profile>
      <id>myrepo</id>
      <activation>
        <activeByDefault>true</activeByDefault>
      </activation>
      <repositories>
        <repository>
          <id>myrepo-read</id>
          <url>https://maven.mydomain.org/</url>
        </repository>
      </repositories>
    </profile>
  </profiles>

</settings>

 

Using an Association to a data (contact) list

The aim of this post is to describe how to use associations in Alfresco to link to a data list.

The use case I’m addressing here is to be able to associate a folder with people held in a contact list.

The first thing to do is to create the association in the model.

Here I’m creating an aspect that allows for one primary contact and n other contacts.

You’ll notice that there are properties as well as associations – more about that later.

        <aspect name="ael:contacts">
            <title>Contacts</title>
            <properties>
                <property name="ael:primaryContact">
                    <title>Primary Contact</title>
                    <type>d:text</type>
                    <mandatory>false</mandatory>
                    <multiple>false</multiple>
                </property>
                <property name="ael:otherContact">
                    <title>Other Contact</title>
                    <type>d:text</type>
                    <mandatory>false</mandatory>
                    <multiple>true</multiple>
                </property>

            </properties>
            <associations>
                <association name="ael:mainContact">
                    <source>
                        <mandatory>false</mandatory>
                        <many>true</many>
                    </source>
                    <target>
                        <class>dl:contact</class>
                        <mandatory>false</mandatory>
                        <many>false</many>
                    </target>
                </association>
                <association name="ael:otherContacts">
                    <source>
                        <mandatory>false</mandatory>
                        <many>true</many>
                    </source>
                    <target>
                        <class>dl:contact</class>
                        <mandatory>false</mandatory>
                        <many>true</many>
                    </target>
                </association>
            </associations>
        </aspect>
    </aspects>

You’ll need to create a data list to link to so let’s create a contact list called External Contacts.

Now it’s time to set up the edit form in the share-config-custom.xml so add the following to the appearance section of the relevant form

<field id="ael:mainContact" label-id="ael.metadata.mainContact"
                        set="aelMetadata">
  <control template="/org/alfresco/components/form/controls/association.ftl">
     <control-param name="startLocation">//cm:External_Contacts</control-param>
  </control>
</field>
<field id="ael:otherContacts" label-id="ael.metadata.otherContact"
                        set="aelMetadata">
  <control 
     template="/org/alfresco/components/form/controls/association.ftl">
     <control-param name="startLocation">//cm:External_Contacts</control-param>
  </control>
</field>

Now this is where it starts to get interesting….

You’ll notice that the start location is an xpath string to //cm:External_Contacts whereas our contact list is called ‘External Contacts’. If you run this then the xpath won’t be found so you’ll taken to the root – so navigate down to the site and into data lists and you’ll see that all the nodes are represented by their uid – not very helpful so let’s go in and change the name of the contact list using a script (the excellent javascript console is useful here)

document.name = 'External_Contacts';
document.save();

Note that it’s document.name not document.properties.name

(You won’t see the underscore and it makes the xpath easier)

So now if you run it you’ll be taken to the right place but any contacts will show up as uids again – so another script this time tied into the create and update actions on the folder.

 var first = document.properties['dl:contactFirstName'];
var last = document.properties['dl:contactLastName'];
var displayName = "";
if (last != null && last.length > 0) {
  displayName = last;
} else if (first != null && first.length > 0) {
  displayName = first;
}
if (first != null && first.length > 0 && last != null && last.length > 0) {
 displayName = last + ", " + first;
}
document.properties.name = displayName;
document.save();

Now we’re at the point where we can edit the properties and usefully create an association with a member of the contact list.

Next adding a custom advanced search – simple just use the association control in the search form and we’re away – NO – it looks like it works, you get the selection dialogs coming up but no results are returned. Unfortunately the Alfresco search doesn’t work across associations so now it’s time to get a bit creative and use those extra properties we set up earlier.

So on your folder with the aspect applied it’s time to add another action using the script below

for each (var assoc in document.assocs["ael:mainContact"]) {
  document.properties["ael:primaryContact"] = assoc.properties["cm:name"];
}

var contacts = new Array();
for each (var assoc in document.assocs["ael:otherContacts"]) {
  contacts.push(assoc.properties["cm:name"]);
}
document.properties["ael:otherContact"] = contacts;

document.save();

Once you’ve done this then you can use the otherContact and primaryContact fields in your search form instead of the mainContact and otherContacts fields. This means that you’ll have text searches and won’t get the option to select your contacts but at least it works and does allow the use of wildcard searching.

Another advantage of having these fields available is that you can use them in DocumentList renderer.

Prior to having these fields I created a custom renderer to follow the associations and get the properties to display. This worked but was a bit overcomplex (it would have helped if the propertyName was available to the renderer function so that the same function could be used for different properties – I used a hack with the label to achieve the same effect)

I discovered a draw back to this approach – the cm:name has to be a valid filename so something like Bloggs, Joe M.I. isn’t valid

 

CMIS using perl

I’ve had a requirement to get some document details out of Alfresco using perl.

CMIS seems to be the obvious answer and a little bit of googling gives us the WebService::Cmis package.

This seems to do the job nicely but the documentation isn’t great (hint: use the pod) so here is a trival example to recurse through a folder and print out the details of the contents.

#!/usr/bin/perl -w
use Data::Dumper;
use WebService::Cmis;
use Cache::FileCache;

print "Content-type:text/htmlrnrn";

my $client = WebService::Cmis::getClient(
url => "https://myhost/alfresco/cmisatom",
user => '',
password => "",
cache => new Cache::FileCache({
cache_root => "/tmp/cmis_client"
}
)
);

my $repo = $client->getRepository;
# print Dumper($repo);

my $projectFolder = "/Sites/mySite/documentLibrary/myFolder";

my $folder = $repo->getObjectByPath($projectFolder);
# print Dumper($folder);
showFolderContents($folder);

 

 

sub showFolderDetails {
 my $fold = shift;
 print "<h2>".$fold->getTitle()."</h2>n";
# my $props = $fold->getProperties;
#print Dumper($props);
}

sub showDocumentDetails {
 my $doc = shift;
# print Dumper($doc);
 my $props = $doc->getProperties;
 if ($props->{'cmis:isLatestVersion'}->getValue eq 1) {
  #Show mail messages
  if (defined $props->{'imap:messageFrom'}) {
   print $props->{'imap:messageFrom'}->getValue;
   print $props->{'imap:messageTo'}->getValue;
   print $props->{'imap:messageSubject'}->getValue;
   print $props->{'imap:flagAnswered'}->getValue;
  } else {
   print $doc->getTitle()."n";
  }
# print Dumper($props);
 }
}

sub showFolderContents {
 my $fold = shift;

 showFolderDetails($fold);
 my $projects = $fold->getChildren();
 while (($entry = $projects->getNext())){

   if ($entry->isa("WebService::Cmis::Folder")) {
      showFolderContents($entry);
   } else {
      showDocumentDetails($entry);
   }
 }
}

 

How to tell if my DHCP ipaddress has changed

A line of script to send you an email if your DHCP allocated IP address has changed.

Either run it from a cron or put it in /etc/rc.local

ls -rt /var/lib/dhcp/dhclient*eth0* | xargs grep fixed-address | tail -2 | awk ‘{print $3}’ | xargs echo -n | sed -e ‘s/;//g’ | awk ‘{if ($1 != $2) { print $2}}’ | mail me@example.org -E’set nonullbody’ -s “My new IP”

Spring Roo ConverterNotFound

In webmvc-config.xml look for

<bean class="org.wwarn.cb.web.ApplicationConversionServiceFactoryBean" id="applicationConversionService"/>

This will give you the name of the class to edit so go and find that class and add the following code:

Note that the installFormatters method is deprecated in 3.1 but as that is what Roo is currently generating I’m leaving it alone

public Converter getStudyConverter() {
        return new Converter() {
            public String convert(ChassisStudies source) {
                return source.getStudyId();
            }
        };
    }
	@Override
	protected void installFormatters(FormatterRegistry registry) {
		super.installFormatters(registry);
		// Register application converters and formatters
		registry.addConverter(getStudyConverter());
	}

MySQL performance problems

I wrote a stored procedure on MySQL that is using a cursor to update a table of about 45,000 rows.

The details what it is doing is are unimportant but the logic is very simple.

The procedure was running extremely slowly (of the order of 1/10 th second per operation) – the steps below describe what I did to make sure that the procedure ran reasonably quickly.

Taking a look in mysql.log I saw the message:

120329 10:34:48 InnoDB: ERROR: the age of the last checkpoint is 9433926,
InnoDB: which exceeds the log group capacity 9433498.
InnoDB: If you are using big BLOB or TEXT rows, you must set the
InnoDB: combined size of log files at least 10 times bigger than the
InnoDB: largest such row.

This leads me to changing my.ini to add (or change) the value of innodb_log_file_size
(Should really read the manual….)


innodb_log_file_size = 64M

You then need to shutdown the server and (re)move the existing log files(ib_logfile0 and ib_logfile1) before starting again with the new values
Official docs
Another value that looks to be recommended to change is innodb_buffer_pool_size and while I was there I changed some other values:

innodb_open_files = 512
#innodb_data_file_path=ibdata4:2G:autoextend:max:4G
innodb_buffer_pool_size = 512M
#innodb_buffer_pool_size = 4G
innodb_additional_mem_pool_size = 512M
innodb_log_file_size = 64M
innodb_log_buffer_size = 8M
innodb_thread_concurrency = 8
innodb_concurrency_tickets = 500
innodb_lock_wait_timeout=200
#Not windows
#innodb_flush_method = O_DIRECT
innodb_autoinc_lock_mode= 2
innodb_commit_concurrency=4
innodb_flush_log_at_trx_commit=2
innodb_support_xa=false
innodb_checksums=0
innodb_doublewrite=0
innodb_max_dirty_pages_pct=15
#innodb_io_capacity = 10000
#innodb_adaptive_checkpoint = 1
#innodb_write_io_threads = 8
#innodb_read_io_threads = 8

 

Other tips

innotop is useful to see what is happening

If there are (slow) temporary tables being created (EXPLAIN Using where; Using temporary; Using filesort ) then try:

mount a tmpfs system on an empty directory (you should also add this to fstab):
mount tmpfs /tmpfs -t tmpfs
and edit my.cnf to make MySQL use that directory as a temporary directory:
tmpdir = /tmpfs

Alfresco CMIS host name

The problem we’ve been having is to do with our Alfresco server name when using CMIS.

We are running our Alfresco instance behind an Apache server (on a different box) and the default configuration means that when we make a call to the CMIS server all the links returned have the link to the Alfresco host which we can’t follow as it’s behind a firewall.

ApacheHost.org -> AlfrescoHost.org

e.g.  Make a call to https://ApacheHost.org/alfresco/service/cmis and all the links come back with AlfrescoHost.org as part of the URL

The solution described here Alfresco JIRA issue is as follows:

Add to the file

tomcat/shared/classes/META-INF/spring-webscripts-config-custom.xml

(You may need to create the file)

 <alfresco-config>

    <config evaluator="string-compare" condition="Server">
       <server>
          <scheme>http</scheme>
          <hostname>your-public-host</hostname>
          <port>your-port</port>
          <path>alfresco</path>
       </server>
    </config>

 </alfresco-config>

 

If you haven’t done it already then it’s probably worth adding/changing the following in alfresco-global.properties as well

alfresco.context=alfresco
alfresco.host=${localname}
alfresco.port=8080
alfresco.protocol=http

share.context=share
share.host=${localname}
share.port=8080
share.protocol=http