Configuring Liferay shard

Goal

Configuring a Liferay in shard mode

Description

This recipe explains the basic steps to configure a Liferay installation in shard mode. In the recipe explanation we will assume a complete fresh installation of Liferay portal, version 6.1.2 CE, using MySQL database.

How to

The recipe consists on the following basic steps:

  • Create the shard databases (lportaldefault and lportalshard1)
  • Configuring portal-ext.properties (or portal-setup-wizard.properties, if you prefer to configure it that way)
  • Configuring the file shard-data-source-spring.xml appropriately

For more informations, you may consult the official information in Lifray’s site.

As for the first step, we will create two schemas: one for the default shard and another for the specific shard:

mysqladmin -uroot create lportaldefault;
mysqladmin -uroot create lportalshard1;

Next, we will configure the portal-ext.properties (which must be within $LIFERAY_HOME) as follows (only the relevant parts are shown):

# Notice that META-INF/shard-data-source-spring.xml must be uncommented for database sharding
spring.configs=META-INF/base-spring.xml,META-INF/hibernate-spring.xml,META-INF/infrastructure-spring.xml,\
META-INF/management-spring.xml,META-INF/util-spring.xml,META-INF/jpa-spring.xml,META-INF/executor-spring.xml,\
META-INF/audit-spring.xml,META-INF/cluster-spring.xml,META-INF/editor-spring.xml,META-INF/jcr-spring.xml,\
META-INF/ldap-spring.xml,META-INF/messaging-core-spring.xml,META-INF/messaging-misc-spring.xml,\
META-INF/mobile-device-spring.xml,META-INF/notifications-spring.xml,META-INF/poller-spring.xml,\
META-INF/rules-spring.xml,META-INF/scheduler-spring.xml,META-INF/scripting-spring.xml,META-INF/search-spring.xml,\
META-INF/workflow-spring.xml,META-INF/counter-spring.xml,META-INF/mail-spring.xml,META-INF/portal-spring.xml,\
META-INF/portlet-container-spring.xml,META-INF/staging-spring.xml,META-INF/virtual-layouts-spring.xml,\
META-INF/dynamic-data-source-spring.xml,META-INF/shard-data-source-spring.xml,#META-INF/memcached-spring.xml,\
#META-INF/monitoring-spring.xml,classpath*:META-INF/ext-spring.xml

shard.default.name=default

# Use ManualShardSelector if you want to choose where to put a specific portal instance data
#shard.selector=com.liferay.portal.dao.shard.RoundRobinShardSelector
shard.selector=com.liferay.portal.dao.shard.ManualShardSelector

##
## JDBC - default shard
##
jdbc.default.driverClassName=com.mysql.jdbc.Driver
jdbc.default.username=root
jdbc.default.password=
jdbc.default.url=jdbc:mysql://localhost/lportaldefault?useUnicode=true&characterEncoding=UTF-8&useFastDateParsing=false

##
## JDBC - for database sharding (use as many as the number of shards you want to create)
## The name (shard1) will have to be the same as the one configured in shard-data-source-spring.xml
##
jdbc.shard1.driverClassName=com.mysql.jdbc.Driver
jdbc.shard1.url=jdbc:mysql://localhost/lportalshard1?useUnicode=true&characterEncoding=UTF-8&useFastDateParsing=false
jdbc.shard1.username=root
jdbc.shard1.password=

And finally, we need to configure the file shard-data-source-spring.xml where we will specify the shard datasources, which must be included in $LIFERAY_HOME/tomcat-7.0.40/webapps/ROOT/WEB-INF/classes/META-INF/ (all content is shown so that you may use a complete/ready file – change only the parts in bold, according to your needs):

<?xml version="1.0"?>

<beans 
  default-destroy-method="destroy" 
  default-init-method="afterPropertiesSet"
  xmlns="http://www.springframework.org/schema/beans"
  xmlns:aop="http://www.springframework.org/schema/aop"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/aop 
      http://www.springframework.org/schema/aop/spring-aop-3.0.xsd 
      http://www.springframework.org/schema/beans 
      http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">

  <bean id="liferayDataSource" class="org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy">
    <property name="targetDataSource">
      <bean class="org.springframework.aop.framework.ProxyFactoryBean">
        <property name="targetSource" ref="shardDataSourceTargetSource" />
      </bean>
    </property>
  </bean>

  <bean id="liferayHibernateSessionFactory" class="org.springframework.aop.framework.ProxyFactoryBean">
    <property name="targetSource" ref="shardSessionFactoryTargetSource" />
  </bean>

  <!-- default shard -->
  <bean id="shardDataSource0" class="org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy">
    <property name="targetDataSource">
      <bean class="com.liferay.portal.dao.jdbc.spring.DataSourceFactoryBean">
        <property name="propertyPrefix" value="jdbc.default." />
      </bean>
    </property>
  </bean>

  <!-- shard1 - create as many as necessary -->
  <bean id="shardDataSource1" class="org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy">
    <property name="targetDataSource">
      <bean class="com.liferay.portal.dao.jdbc.spring.DataSourceFactoryBean">
        <property name="propertyPrefix" value="jdbc.shard1." />
      </bean>
    </property>
  </bean>

  <bean id="shardDataSourceTargetSource" class="com.liferay.portal.dao.shard.ShardDataSourceTargetSource">
    <property name="dataSources">
      <map>
        <entry>
          <key>
            <value>default</value>
          </key>
          <ref bean="shardDataSource0" />
        </entry>
        <entry>
          <key>
            <value>shard1</value>
          </key>
          <ref bean="shardDataSource1" />
        </entry>
      </map>
    </property>
  </bean>
  
  <bean id="shardSessionFactoryTargetSource" 
        class="com.liferay.portal.dao.shard.ShardSessionFactoryTargetSource">
    <property name="shardDataSourceTargetSource" ref="shardDataSourceTargetSource" />
  </bean>
  <bean id="com.liferay.portal.kernel.dao.jdbc.MappingSqlQueryFactoryUtil" 
        class="com.liferay.portal.kernel.dao.jdbc.MappingSqlQueryFactoryUtil">
    <property name="mappingSqlQueryFactory">
      <bean class="com.liferay.portal.dao.shard.ShardMappingSqlQueryFactoryImpl" />
    </property>
  </bean>
  <bean id="com.liferay.portal.kernel.dao.jdbc.SqlUpdateFactoryUtil" 
        class="com.liferay.portal.kernel.dao.jdbc.SqlUpdateFactoryUtil">
    <property name="sqlUpdateFactory">
      <bean class="com.liferay.portal.dao.shard.ShardSqlUpdateFactoryImpl" />
    </property>
  </bean>
  <bean id="com.liferay.portal.kernel.dao.shard.ShardUtil" 
        class="com.liferay.portal.kernel.dao.shard.ShardUtil">
    <property name="shard">
      <bean class="com.liferay.portal.dao.shard.ShardImpl">
        <property name="shardAdvice" ref="com.liferay.portal.dao.shard.advice.ShardAdvice" />
      </bean>
    </property>
  </bean>
  <bean id="com.liferay.portal.kernel.util.InfrastructureUtil" 
        class="com.liferay.portal.kernel.util.InfrastructureUtil">
    <property name="dataSource" ref="liferayDataSource" />
    <property name="mailSession" ref="mailSession" />
    <property name="shardDataSourceTargetSource" ref="shardDataSourceTargetSource" />
    <property name="shardSessionFactoryTargetSource" ref="shardSessionFactoryTargetSource" />
    <property name="transactionManager" ref="liferayTransactionManager" />
  </bean>
  <bean id="com.liferay.portal.dao.shard.advice.ShardAdvice" 
        class="com.liferay.portal.dao.shard.advice.ShardAdvice">
    <property name="shardDataSourceTargetSource" ref="shardDataSourceTargetSource" />
    <property name="shardSessionFactoryTargetSource" ref="shardSessionFactoryTargetSource" />
  </bean>
  <bean id="com.liferay.portal.dao.shard.advice.ShardCompanyAdvice" class="com.liferay.portal.dao.shard.advice.ShardCompanyAdvice">
    <property name="shardAdvice" ref="com.liferay.portal.dao.shard.advice.ShardAdvice" />
  </bean>
  <bean id="com.liferay.portal.dao.shard.advice.ShardPortletAdvice" class="com.liferay.portal.dao.shard.advice.ShardPortletAdvice">
    <property name="shardAdvice" ref="com.liferay.portal.dao.shard.advice.ShardAdvice" />
  </bean>
  <bean id="com.liferay.portal.dao.shard.advice.ShardPersistenceAdvice" class="com.liferay.portal.dao.shard.advice.ShardPersistenceAdvice">
    <property name="shardAdvice" ref="com.liferay.portal.dao.shard.advice.ShardAdvice" />
  </bean>
  <bean id="com.liferay.portal.dao.shard.advice.ShardGloballyAdvice" class="com.liferay.portal.dao.shard.advice.ShardGloballyAdvice">
    <property name="shardAdvice" ref="com.liferay.portal.dao.shard.advice.ShardAdvice" />
  </bean>
  <bean id="com.liferay.portal.dao.shard.advice.ShardIterativelyAdvice" class="com.liferay.portal.dao.shard.advice.ShardIterativelyAdvice">
    <property name="shardAdvice" ref="com.liferay.portal.dao.shard.advice.ShardAdvice" />
  </bean>
  <bean id="com.liferay.portal.dao.shard.advice.ShardParameterAdvice" class="com.liferay.portal.dao.shard.advice.ShardParameterAdvice">
    <property name="shardAdvice" ref="com.liferay.portal.dao.shard.advice.ShardAdvice" />
  </bean>
  <aop:config proxy-target-class="false">
    <aop:advisor advice-ref="com.liferay.portal.dao.shard.advice.ShardCompanyAdvice" 
         pointcut="bean(com.liferay.portal.service.CompanyLocalService)" />
    <aop:advisor advice-ref="com.liferay.portal.dao.shard.advice.ShardPortletAdvice" 
         pointcut="bean(com.liferay.portal.service.PortletLocalService)" />
    <aop:advisor advice-ref="com.liferay.portal.dao.shard.advice.ShardPersistenceAdvice" 
         pointcut="bean(*Persistence) || bean(*Finder)" />
    <aop:advisor advice-ref="com.liferay.portal.dao.shard.advice.ShardGloballyAdvice" 
         pointcut="execution(void com.liferay.portal.convert.messaging.ConvertProcessMessageListener.receive(..))" />
    <aop:advisor advice-ref="com.liferay.portal.dao.shard.advice.ShardGloballyAdvice" 
         pointcut="execution(void com.liferay.portal.events.StartupHelper.updateIndexes())" />
    <aop:advisor advice-ref="com.liferay.portal.dao.shard.advice.ShardGloballyAdvice" 
         pointcut="execution(void com.liferay.portal.events.StartupHelper.upgradeProcess(int))" />
    <aop:advisor advice-ref="com.liferay.portal.dao.shard.advice.ShardIterativelyAdvice" 
         pointcut="execution(void com.liferay.portal.events.StartupHelper.verifyProcess(boolean))" />
    <aop:advisor advice-ref="com.liferay.portal.dao.shard.advice.ShardGloballyAdvice" 
         pointcut="execution(* com.liferay.portal.service.ReleaseLocalService.getBuildNumberOrCreate())" />
    <aop:advisor advice-ref="com.liferay.portal.dao.shard.advice.ShardGloballyAdvice" 
         pointcut="execution(void com.liferay.portal.service.ServiceComponentLocalService.upgradeDB(..))" />
    <aop:advisor advice-ref="com.liferay.portal.dao.shard.advice.ShardGloballyAdvice" 
         pointcut="execution(void com.liferay.portlet.journal.service.JournalArticleLocalService.checkArticles())" />
    <aop:advisor advice-ref="com.liferay.portal.dao.shard.advice.ShardParameterAdvice" 
         pointcut="execution(* com.liferay.portal.service.AccountLocalService.getAccount(long, long))" />
    <aop:advisor advice-ref="com.liferay.portal.dao.shard.advice.ShardParameterAdvice" 
         pointcut="execution(* com.liferay.portal.service.ResourceCodeLocalService.checkResourceCodes(long, String))" />
    <aop:advisor advice-ref="com.liferay.portal.dao.shard.advice.ShardParameterAdvice" 
         pointcut="execution(* com.liferay.portal.service.UserLocalService.searchCount(..))" />
  </aop:config>
</beans>

Explanations

At this point, you may start your Liferay portal and, on the creation of a portal instance, you have an option to add the shard you want to use for that portal instance. After configuring that portal instance and accessing it, you will realize that your data is stored on the configured shard and not on the default database.

Advertisements

3 comments

  1. I do not have mentioned .xml in $LIFERAY_HOME/tomcat-7.0.40/webapps/ROOT/WEB-INF/classes/META-INF – Liferay 6.2.1

    It is not necessary to create an ext plugin in order to modify those names?

    1. Hello Michal,

      Yes, I suppose you don’t have that file yet. That’s why I show a complete one in this post and tell you to put it in your tomcat’s folder. Notice also that the path for Liferay 6.2 shouldn’t be the one I stated because your Tomcat version will probably be different. Hope this helps.

      Regarding the “ext plugin in order to modify those names”, I don’t think I understood what you mean…

  2. i have configured everything according to an article, i can choose a shard manually for my instance, but anyway, only default database is used, database1 and database2 are empty. May be you can advise, thanks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s