Tag: drupal

Testing a website from server before DNS switch via CURL

If we come across a scenario where we are migrating a high traffic site from a third party hosting provider like hostgator or godaddy to a IaaS host, we will come across a situation where we want to test if the site that we setup on the IaaS server is ready for us to do a DNS switch.

To accomplish that testing we can use ssh to the IaaS server and once all is setup we can test it using the curl command as below:

curl --verbose --header 'Host: www.mywebsite.com' https://ipaddress:port -k

–verbose will give you a detailed overview of the command

–header ‘Host: ‘ will allow us to tell apache2 which domain name are we trying to test.

Then we provide the https://[current ip]:[port] to tell curl that we want to check the response for the above domain name on this ip address and port.

-k will allow curl to ignore any SSL certificate issues if you havent finished transferring the SSL certificates to the server yet.

Apache2 Alias directive in different virtual hosts files

Situation: 

I have two Drupal / PHP websites / applications

https://qa.example.com/app1

https://qa.example.com/app2

I wanted to setup Apache2 to listen for SSL connection on port 9443 and need to have 2 seperate virtual host files or site files under “sites-available” folder in Apache2.

I created something like this for each

<IfModule mod_ssl.c>
<VirtualHost *:9443>
ServerName qa.example.com
ServerAdmin webmaster@server

Alias /app1 /var/www/html/qa/app1
DocumentRoot “/var/www/html/qa/app1”

# Available loglevels: trace8, …, trace1, debug, info, notice, warn,
# error, crit, alert, emerg.
# It is also possible to configure the loglevel for particular
# modules, e.g.
#LogLevel info ssl:warn

ErrorLog ${APACHE_LOG_DIR}/app1-ssl-qa-error.log
CustomLog ${APACHE_LOG_DIR}/app1-ssl-qa-access.log combined

# For most configuration files from conf-available/, which are
# enabled or disabled at a global level, it is possible to
# include a line for only one particular virtual host. For example the
# following line enables the CGI configuration for this host only
# after it has been globally disabled with “a2disconf”.
#Include conf-available/serve-cgi-bin.conf

# SSL Engine Switch:
# Enable/Disable SSL for this virtual host.
SSLEngine on

# A self-signed (snakeoil) certificate can be created by installing
# the ssl-cert package. See
# /usr/share/doc/apache2/README.Debian.gz for more info.
# If both key and certificate are stored in the same file, only the
# SSLCertificateFile directive is needed.
SSLCertificateFile /etc/ssl/certs/example.pem
SSLCertificateKeyFile /etc/ssl/private/example.key

<FilesMatch “\.(cgi|shtml|phtml|php)$”>
SSLOptions +StdEnvVars
</FilesMatch>
# <Directory /usr/lib/cgi-bin>
<Directory “/var/www/html/qa/app1”>
SSLOptions +StdEnvVars
Options Indexes FollowSymLinks MultiViews
AllowOverride All
</Directory>

</VirtualHost>
</IfModule>

The issue: 

I need Apache to load App1 when app1 url is accessed and app2 when app2 url is accessed. But when setting up the two apps with seperate virtual host files like above, accessing the apps using either urls only loaded the first virtual host listed in Apache.

The Fix:

I had to merge the two aliases to one virtual hosts file like below to make it work how I want.

 

<IfModule mod_ssl.c>
<VirtualHost *:9443>
ServerName qa.example.com
ServerAdmin webmaster@server

Alias /app1 /var/www/html/qa/app1
DocumentRoot “/var/www/html/qa/app1”

Alias /app2 /var/www/html/qa/app2
DocumentRoot “/var/www/html/qa/app2”

# Available loglevels: trace8, …, trace1, debug, info, notice, warn,
# error, crit, alert, emerg.
# It is also possible to configure the loglevel for particular
# modules, e.g.
#LogLevel info ssl:warn

ErrorLog ${APACHE_LOG_DIR}/app1-ssl-qa-error.log
CustomLog ${APACHE_LOG_DIR}/app1-ssl-qa-access.log combined

# For most configuration files from conf-available/, which are
# enabled or disabled at a global level, it is possible to
# include a line for only one particular virtual host. For example the
# following line enables the CGI configuration for this host only
# after it has been globally disabled with “a2disconf”.
#Include conf-available/serve-cgi-bin.conf

# SSL Engine Switch:
# Enable/Disable SSL for this virtual host.
SSLEngine on

# A self-signed (snakeoil) certificate can be created by installing
# the ssl-cert package. See
# /usr/share/doc/apache2/README.Debian.gz for more info.
# If both key and certificate are stored in the same file, only the
# SSLCertificateFile directive is needed.
SSLCertificateFile /etc/ssl/certs/example.pem
SSLCertificateKeyFile /etc/ssl/private/example.key

<FilesMatch “\.(cgi|shtml|phtml|php)$”>
SSLOptions +StdEnvVars
</FilesMatch>
# <Directory /usr/lib/cgi-bin>
<Directory “/var/www/html/qa/app1“>
SSLOptions +StdEnvVars
Options Indexes FollowSymLinks MultiViews
AllowOverride All
</Directory>

<Directory “/var/www/html/qa/app2“>
SSLOptions +StdEnvVars
Options Indexes FollowSymLinks MultiViews
AllowOverride All
</Directory>

</VirtualHost>
</IfModule>

 

This way the problem is that we cannot disable one app and keep the other enabled. If anyone knows a solution for this please let me know.

PHP 7.0 and Drupal 7 on Ubuntu 16.04 Xenial

Ubuntu 16.04 doesn’t natively let you install PHP5.6. It only lets you install PHP 7.0. It is a long process to install PHP 5.6 if your server is not allowed to connect to the internet.

Drupal 7, as of Drupal 7.44, (approximately July 2016) supports and works with PHP 7.0 .So you may install PHP 7.0 on Ubuntu 16.04 for Drupal 7 sites instead of jumping through the hoops to get PHP5.6 installed on the server.

 

Reference: Ubuntu Drupal 7 package 

Git Prune for Cleanup

When you or a team member removes a few branch from your remote git repository as part of cleanup and you still see the branches on your local environment you will have to run the git command below to remove (prune) all the branches from your local that do not exist on the remote repository anymore.

git fetch -p

Setting up the simplenews module for bulk email sending in drupal

In the past, I had a very bad experience coding a PHP script to send bulk e-mail. This time we needed a newsletter sending module for the drupal site to send around 55,000 newsletter emails.

For this I found the simplenews module on the drupal site that served the basic purpose of a newsletter sending application. As per our needs, I also installed the other modules/plugins like simplenews template, simplenews statistics and mime mail which enhanced the features of the simplenews module.

The modules, plugins and other requirements:

The simplenews module: This is the core module needed for the newsletter functionality

New Content type: Create a content type for the simplenews module to use

The simplenews template module: This was needed to add custom header and footer to content type created for simplenews on drupal so that we could send a particular webpage/node (special content type for simplenews) as the newsletter.

The statistics module: This was needed for reporting purposes. It gives the information about how many people clicked the links on the mail and viewed the email etc..

Mime mail: This module helps in sending HTML mails and is needed to be used with simplenews template.

The installation and configuration:

First I installed the simplenews module and then the simplenews template and statistics (no specific order needed) then the mime mail module to send out HTML format emails.

Created a content type for the simplenews module and setup the taxonomy and pathauto settings for the terms for the content type so that once a node or page is created the link for it is properly generated.

The installation of “mime mail css compressor” needed the DOM php plugin installed so I did a yum install on the php plugins on the linux server and it updated the needed plugins for installation of css compressor.

Once the installation of all the required modules were done. I configured the simplenews module. One of the most important settings was to setup the mail send process to be run using the CRON job, and to setup such that it send more messages per cron run that doesn’t let the cron timeout. The max number of messages per cron job that I could successfully send were 200. setting it to 500 failed the cron run due to timeout.

Our server was setup to run cronjobs every one hour. This was very unfavourable for the simplenews module to send 55,000 messages. It would take 275 hours to finish the job in this case. Therefore I did a little research and came across ELYSIA cron module which splits all the cron jobs in drupal and lets us setup different cron run timings for different modules.

I changed the crontab drupal maintenance period to * * * * * and then in the elysia cron module settings changed the schedule for the simplenews module to be run every 5 minutes (*/5 * * * *) . Now the simplenews module would take only 23 hours to finish sending all the messages. (Each cron run was taking on an average 5 minutes to complete hence the time was selected as 5 mins)

Now the send process of the mail takes its time to send all the messages without crashing the smtp server and without backfiring and sending multiple mails to same user. This is the solution to sending bulk newsletters without crashing anything.

If you have any questions, please feel free to ask me.