Author: Aaron Belchamber

Senior Cloud Software Engineer and 25+ years experienced video production, video editing and 3D animation services for a variety of global clients including local video production here in Jacksonville, Florida.

Updating to Mautic 2.9.1 mild issues resolved

07/31/2017 Aaron Belchamber No Comments

As a PHP web developer with over 13 years experience, I can say that Mautic is the best of all the free marketing automation platforms, by far.  Sugar CRM is good, for what it does, but that’s a CRM, not an MA (Marketing Automation).  Still, when you are updating Mautic, like you would WordPress, be sure to always back up your files and database!

Now, I will chronicle my latest experience updating Mautic from 2.8.3 and 2.9.1.  By clicking the update button beneath the bell icon, it take you to the update page where you have to click “Update”.  Then, the process unfolds, it seems to hang at clearing the cache on my server, but you have to logout then log back in.  Everything went fine, except when I went to “Emails” and “Focus Items” and it hung up.  In the logs, I saw the following entries:

mautic.CRITICAL: Uncaught PHP Exception Doctrine\DBAL\Exception\InvalidFieldNameException: "An exception occurred while executing 'SELECT COUNT(*) AS dctrn_count FROM (SELECT DISTINCT id_10 FROM (SELECT e0_.is_published AS is_published_0, e0_.date_added AS date_added_1, e0_.created_by AS created_by_2, e0_.created_by_user AS created_by_user_3, e0_.date_modified AS date_modified_4, e0_.modified_by AS modified_by_5, e0_.modified_by_user AS modified_by_user_6, e0_.checked_out AS checked_out_7, e0_.checked_out_by AS checked_out_by_8, e0_.checked_out_by_user AS checked_out_by_user_9, e0_.id AS id_10, e0_.name AS name_11, e0_.description AS description_12, e0_.subject AS subject_13, e0_.from_address AS from_address_14, e0_.from_name AS from_name_15, e0_.reply_to_address AS reply_to_address_16, e0_.bcc_address AS bcc_address_17, e0_.template AS template_18, e0_.content AS content_19, e0_.utm_tags AS utm_tags_20, e0_.plain_text AS plain_text_21, e0_.custom_html AS custom_html_22, e0_.email_type AS email_type_23, e0_.publish_up AS publish_up_24, e0_.publish_down AS publish_down_25, e0_.read_count AS read_count_26, e0_.sent_count AS sent_count_27, e0_.revision AS revision_28, e0_.lang AS lang_29, e0_.variant_settings AS variant_settings_30, e0_.variant_start_date AS variant_start_date_31, e0_.dynamic_content AS dynamic_content_32, e0_.variant_sent_count AS variant_sent_count_33, e0_.variant_read_count AS variant_read_count_34 FROM emails e0_ LEFT JOIN categories c1_ ON e0_.category_id = c1_.id WHERE e0_.variant_parent_id IS NULL AND e0_.translation_parent_id IS NULL ORDER BY e0_.subject DESC) dctrn_result) dctrn_table':  SQLSTATE[42S22]: Column not found: 1054 Unknown column 'e0_.utm_tags' in 'field list'" at /home/marketingautomation/yourdirector/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php line 71 {"exception":"[object] (Doctrine\\DBAL\\Exception\\InvalidFieldNameException(code: 0): An exception occurred while executing 'SELECT COUNT(*) AS dctrn_count FROM (SELECT DISTINCT id_10 FROM (SELECT e0_.is_published AS is_published_0, e0_.date_added AS date_added_1, e0_.created_by AS created_by_2, e0_.created_by_user AS created_by_user_3, e0_.date_modified AS date_modified_4, e0_.modified_by AS modified_by_5, e0_.modified_by_user AS modified_by_user_6, e0_.checked_out AS checked_out_7, e0_.checked_out_by AS checked_out_by_8, e0_.checked_out_by_user AS checked_out_by_user_9, e0_.id AS id_10, e0_.name AS name_11, e0_.description AS description_12, e0_.subject AS subject_13, e0_.from_address AS from_address_14, e0_.from_name AS from_name_15, e0_.reply_to_address AS reply_to_address_16, e0_.bcc_address AS bcc_address_17, e0_.template AS template_18, e0_.content AS content_19, e0_.utm_tags AS utm_tags_20, e0_.plain_text AS plain_text_21, e0_.custom_html AS custom_html_22, e0_.email_type AS email_type_23, e0_.publish_up AS publish_up_24, e0_.publish_down AS publish_down_25, e0_.read_count AS read_count_26, e0_.sent_count AS sent_count_27, e0_.revision AS revision_28, e0_.lang AS lang_29, e0_.variant_settings AS variant_settings_30, e0_.variant_start_date AS variant_start_date_31, e0_.dynamic_content AS dynamic_content_32, e0_.variant_sent_count AS variant_sent_count_33, e0_.variant_read_count AS variant_read_count_34 FROM emails e0_ LEFT JOIN categories c1_ ON e0_.category_id = c1_.id WHERE e0_.variant_parent_id IS NULL AND e0_.translation_parent_id IS NULL ORDER BY e0_.subject DESC) dctrn_result) dctrn_table':\n\nSQLSTATE[42S22]: Column not found: 1054 Unknown column 'e0_.utm_tags' in 'field list' at /home/marketingautomation/yourdirectory/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php:71, Doctrine\\DBAL\\Driver\\PDOException(code: 42S22): SQLSTATE[42S22]: Column not found: 1054 Unknown column 'e0_.utm_tags' in 'field list' at /home/marketingautomation/yourdirectory/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:106, PDOException(code: 42S22): SQLSTATE[42S22]: Column not found: 1054 Unknown column 'e0_.utm_tags' in 'field list' at /home/marketingautomation/yourdirectory/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:104)"} []

So here’s what I did on the command line to force the Mautic update to finish:

$ app/console mautic:update:find -e prod
PHP Warning:  Module 'XCache' already loaded in Unknown on line 0
Great! You are running the current version of Mautic.
$ php app/console doctrine:migration:status
PHP Warning:  Module 'XCache' already loaded in Unknown on line 0

 == Configuration

    >> Name:                                               Mautic Migrations
    >> Database Driver:                                    pdo_mysql
    >> Database Name:                                      your_database_name
    >> Configuration Source:                               manually configured
    >> Version Table Name:                                 migrations
    >> Version Column Name:                                version
    >> Migrations Namespace:                               Mautic\Migrations
    >> Migrations Directory:                               /home/symfony/yourdirectory/app/migrations
    >> Previous Version:                                   2016-07-26 00:00:00 (20160726000000)
    >> Current Version:                                    2016-07-26 00:00:01 (20160726000001)
    >> Next Version:                                       2016-07-28 00:00:00 (20160728000000)
    >> Latest Version:                                     2017-07-25 15:50:53 (20170725155053)
    >> Executed Migrations:                                37
    >> Executed Unavailable Migrations:                    0
    >> Available Migrations:                               88
    >> New Migrations:                                     51
$ php app/console doctrine:migration:migrate
PHP Warning:  Module 'XCache' already loaded in Unknown on line 0

                    Mautic Migrations


WARNING! You are about to execute a database migration that could result in schema changes and data lost. Are you sure you wish to continue? (y/n)y
Migrating up to 20170725155053 from 20160726000001

  SS skipped (Reason: Schema includes this migration)

  ++ migrating 20160731000000

     -> update campaign_lead_event_log log inner join campaign_events events on log.event_id = events.id set log.metadata = 'a:0:{}' where events.type = 'email.send' and log.metadata = 'a:2:{s:6:"failed";i:1;s:6:"reason";s:50:"mautic.notification.campaign.failed.not_subscribed";}'

  ++ migrated (2.49s)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  ++ migrating 20160926182807

     -> insert into companies (companyname, is_published) (SELECT DISTINCT TRIM(company), 1 from leads l left join companies c ON l.company = c.companyname where company IS NOT NULL and company <> '' and c.companyname is null)
     -> insert into companies_leads (company_id, lead_id, date_added, manually_added, manually_removed) SELECT c.id, l.id, '2017-07-30 12:30:05', 0, 0 from leads l join companies c on c.companyname = l.company ON DUPLICATE KEY UPDATE company_id = c.id;

  ++ migrated (4.85s)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  ++ migrating 20161123225456

Migration 20161123225456 was executed but did not result in any SQL statements.

  ++ migrated (11.54s)

  ++ migrating 20161124145649

     -> ALTER TABLE lead_frequencyrules CHANGE `frequency_number` `frequency_number` SMALLINT DEFAULT NULL;
     -> ALTER TABLE lead_frequencyrules CHANGE `frequency_time` `frequency_time` VARCHAR(25) DEFAULT NULL;

  ++ migrated (2.38s)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  ++ migrating 20170108012944

     -> ALTER TABLE push_notifications CHANGE button button LONGTEXT NULL
     -> ALTER TABLE dynamic_content_lead_data DROP FOREIGN KEY FK_515B221B55458D
     -> ALTER TABLE dynamic_content_lead_data CHANGE lead_id lead_id INT NOT NULL
     -> ALTER TABLE dynamic_content_lead_data ADD CONSTRAINT FK_515B221B55458D FOREIGN KEY (lead_id) REFERENCES leads (id) ON DELETE CASCADE

  ++ migrated (4.52s)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  ++ migrating 20170216221648

     -> ALTER TABLE focus ADD editor LONGTEXT NULL
     -> ALTER TABLE focus ADD html_mode VARCHAR(255) DEFAULT NULL
     -> ALTER TABLE focus ADD html LONGTEXT NULL
     -> ALTER TABLE focus ADD utm_tags LONGTEXT DEFAULT NULL COMMENT '(DC2Type:array)';

  ++ migrated (12.38s)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  ++ migrating 20170429092049

     -> ALTER TABLE emails ADD utm_tags LONGTEXT DEFAULT NULL COMMENT '(DC2Type:array)';
     -> ALTER TABLE push_notifications ADD utm_tags LONGTEXT DEFAULT NULL COMMENT '(DC2Type:array)';

  ++ migrated (4.45s)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  ++ migrating 20170515222314

     -> CREATE TABLE imports (
    id INT AUTO_INCREMENT NOT NULL,
    is_published TINYINT(1) NOT NULL,
    date_added DATETIME DEFAULT NULL COMMENT '(DC2Type:datetime)',
    created_by INT DEFAULT NULL,
    created_by_user VARCHAR(255) DEFAULT NULL,
    date_modified DATETIME DEFAULT NULL COMMENT '(DC2Type:datetime)',
    modified_by INT DEFAULT NULL,
    modified_by_user VARCHAR(255) DEFAULT NULL,
    checked_out DATETIME DEFAULT NULL COMMENT '(DC2Type:datetime)',
    checked_out_by INT DEFAULT NULL,
    checked_out_by_user VARCHAR(255) DEFAULT NULL,
    dir VARCHAR(255) NOT NULL,
    file VARCHAR(255) NOT NULL,
    original_file VARCHAR(255) DEFAULT NULL,
    line_count INT NOT NULL,
    inserted_count INT NOT NULL,
    updated_count INT NOT NULL,
    ignored_count INT NOT NULL,
    priority INT NOT NULL,
    status INT NOT NULL,
    date_started DATETIME DEFAULT NULL COMMENT '(DC2Type:datetime)',
    date_ended DATETIME DEFAULT NULL COMMENT '(DC2Type:datetime)',
    object VARCHAR(255) NOT NULL,
    properties LONGTEXT DEFAULT NULL COMMENT '(DC2Type:json_array)',
    INDEX import_object (object),
    INDEX import_status (status),
    INDEX import_priority (priority),
    PRIMARY KEY(id)
) DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci ENGINE = InnoDB;

  ++ migrated (2.82s)

  ++ migrating 20170516000000

     -> INSERT INTO `lead_fields` (`is_published`, `label`, `alias`, `type`, `field_group`, `default_value`, `is_required`, `is_fixed`, `is_visible`, `is_short_visible`, `is_listable`, `is_publicly_updatable`, `is_unique_identifer`, `field_order`, `properties`, `object`)
VALUES (1, 'Points', 'points', 'number', 'core', '0', 0, 1, 1, 0, 0, 0, 0, 29, 'a:0:{}', 'lead');

  ++ migrated (2.09s)

  ++ migrating 20170517091309

     -> CREATE TABLE lead_event_log (
    id INT AUTO_INCREMENT NOT NULL,
    lead_id INT DEFAULT NULL,
    user_id INT DEFAULT NULL,
    user_name VARCHAR(255) DEFAULT NULL,
    bundle VARCHAR(255) DEFAULT NULL,
    object VARCHAR(255) DEFAULT NULL,
    action VARCHAR(255) DEFAULT NULL,
    object_id INT DEFAULT NULL,
    date_added DATETIME NOT NULL COMMENT '(DC2Type:datetime)',
    properties LONGTEXT DEFAULT NULL COMMENT '(DC2Type:json_array)',
    INDEX lead_id_index (lead_id),
    INDEX lead_object_index (object, object_id),
    INDEX lead_timeline_index (bundle, object, action, object_id),
    INDEX lead_date_added_index (date_added),
    PRIMARY KEY(id)
) DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci ENGINE = InnoDB;
     -> ALTER TABLE lead_event_log ADD CONSTRAINT FK_753AF2E55458D FOREIGN KEY (lead_id) REFERENCES leads (id) ON DELETE SET NULL;

  ++ migrated (5.34s)

  ++ migrating 20170607150241

     -> ALTER TABLE webhook_logs ADD note VARCHAR(255) DEFAULT NULL

  ++ migrated (4.9s)

  ++ migrating 20170607155015

     -> ALTER TABLE webhook_logs ADD runtime DOUBLE PRECISION DEFAULT NULL

  ++ migrated (4.03s)

  SS skipped (Reason: Schema includes this migration)

  SS skipped (Reason: Schema includes this migration)

  ++ migrating 20170628191405

     -> CREATE TABLE plugin_crm_pipedrive_owners (
  id INT AUTO_INCREMENT NOT NULL,
  email VARCHAR(255) NOT NULL,
  owner_id INT DEFAULT NULL,
  INDEX email (email),
  INDEX owner_id (owner_id),
  PRIMARY KEY(id)
) DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci ENGINE = InnoDB;

  ++ migrated (3.61s)

  ++ migrating 20170725155053

     -> ALTER TABLE companies CHANGE score score INT(11) NULL

  ++ migrated (6.32s)

  ------------------------

  ++ finished in 71.72s
  ++ 51 migrations executed
  ++ 23 sql queries

The strange thing is, Mautic says that there has been no database schema changes, except for updates older than 2.6, but I was updating Mautic from the most recent previous verison, 2.8.3.  Oh well, something was missed in translation, can’t really complain about free marketing automation if it has a few glitches once in a while.  I’ve never updated a version of Mautic that was completely smooth but there has always been a workaround.


So you’re an American and you support the H1B Visa program? Watch it — karma will get you next!

07/09/2017 Aaron Belchamber No Comments

Of all people a tech recruiter on LinkedIn recently asked “IT professional”, “Non-IT professionals”, web developers, recruiters, and HR managers what they thought about the H1B Visa Program.

Hmmm.  I should ask him how he feels about cancer or being paid about half of what he’s worth?!  I seldom engage on any social media, but I just couldn’t help but respond:  “It’s disgusting and everyone behind it deserve to have THEIR JOBS depressed with more artificially imported competition, they better watch it because enough good US programmers might find ways to replace them with automation systems. That’s social justice and karma.”

So, a tech recruiter, someone whose job isn’t directly affected by political whims and crony capitalism, is asking “our thoughts” about these destructive, self-serving and anti-American labor policies under the guise of what’s called the H1B Visa Program.  He’s asking as if he really has a dog in the fight, as if his livelihood has been intentionally depressed by supporting the policies of importing substandard IT, programmers and other web professionals and supplanting Americans from jobs they’ve proven are capable of doing.

Many tech recruiters get 20% of many new hires’ first year salary for being a middle man without the same marketable, highly valued skills they are actually selling.  On top of that, their fees price some developers out of good positions, even if the employer finds them most qualified!  Thanks, again recruiters!

So much patronizing arrogance surrounds the tech sector.  So many just don’t understand how we do things, if they don’t get it, it must not be important, this interweb thingy, making it work right and all — many still just don’t value tech.  Many organizations value people with Marketing Degrees a lot more than programmers, database administrators and software engineers.  O, I went to school, got a bachelor’s in Business Management, minored in Marketing but I learned much about the web, including system design, programming and writing code over the span of 15 years.

It’s almost like what we tech people do is so mysterious and nebulous to so many people, we’re not even here.  What people don’t understand, they ignore, they don’t see.  This might be why so many ignore our presence and use that as an excuse to need to import cheap labor knowing it will bring salaries drastically down.  So many people who make their money off the tech industry, like some recruiters, treat web developers and software programmers as commodities instead of people, like cattle instead of fellow countrymen.  Many recruiters are great and try hard for their candidates but shame on those American organizations, businesses and politicians who are exploiting hard working and smart people in their own country, people who spent years learning these highly technical skills, skills admittedly most people just can’t do or don’t have the discipline themselves to study and master.  We’re all neighbors and you basically let your dog crap on our front lawn everyday, how neighborly.

I say since you’re so pro-globalism and want more competition, be careful what you wish for.  I’d like to remind the people who are hurting American workers and their families that their selfish greed will come back because us tech people, programmers, the makers of “internet stuff”, one day we might just all wake up and automate you out of a job, build a better system to compete against your business, make sure you politicians who aren’t protecting our interests that all you selfish H1B’ers out there, the tables will be turned on you.  It’s almost a guarantee, karma has a way of coming around and serving up some delicious humble pie.

Do I sound bitter?  I’m not, a little angry, perhaps because its selfish and unjust for so many who have been affected.  I have friends who lost jobs and had to train their cheaper H1B replacements, just down in Orlando.  Though I know H1B has created competition that has pushed down tech salaries in general, even here as a Director of Web Development in Jacksonville, Florida, I’m pretty grateful I work at a web-centric company that is evolving with its appreciation of web savvy technical expertise and that they value it more than most companies around.

Still, I’m a lucky one and I like to see people be treated fairly and rewarded for their skills, talents and contributions.  A true meritocracy will empower companies that are not just taking part in depressing the tech sector’s wages, they’re really disincentives that are hurting their own bottom line, they’re stifling tech innovation, and that is hurting our economy with billions of dollars worth of misdirected underemployment.  A meritocracy is what capitalism is supposed to model and H1B is an aberration to true market principles because it introduces artificial and unfair pressure to a large labor market with trillions of dollars in costs to our economy, in the form of opportunity costs, misdirected incentives, and resource misallocation.


Upgrading to Drupal 8 looks a lot like Symfony

08/02/2016 Aaron Belchamber No Comments

The good news about upgrading to Drupal 8 is that it looks a lot like Symfony, the bad news is that upgrading to Drupal 8 looks a lot like Symfony.  I guess it’s just your perspective but if you have a lot of sites steeped deeply in D7, it’s quite a transition.  However, if most of your code is a Symfony or other MVC-based framework based web dev shop without much invested in prior versions of Drupal, this transition will not only be easy, you will find your web team able to already recognize, find, and be completely familiar with all the essential organizational patterns under D8’s hood.

Red Crackle has a good list for those looking to transition from Drupal 7 to Drupal 8, just click here to read their list.

Information about migrating from Drupal 7 to Drupal 8 can be found here.

Here’s a useful video/overview of the transition and the major pros and cons.

 

 


Preventing double form submissions with Parsley form validation

07/29/2016 Aaron Belchamber No Comments

Thanks to Jon Leigh for this snippet!  If you use Parsley form validation, a reliable form validation library, here’s an easy way to prevent double submissions of forms.  This disables the form submission button only once the form passes Parsley’s form validation.

$.listen('parsley:form:validated', function(e){
    if (e.validationResult) {
        /* Validation has passed, prevent double submissions */
        $('button[type=submit]').attr('disabled', 'disabled');
  }
});

I’ve implemented this in a few different places and it works flawlessly!  This is especially useful for mobile device users on slow Internet connections where double submissions can be more prevalent.  I hope this helps some businesses out there, it’s helped WordPress Developers and Symfony experts alike improve visitor UX while reducing complaints from people who may have been charged twice.


Mesocolumn WordPress child theme fix after updating theme

05/15/2016 Aaron Belchamber No Comments

It was bound to happen, the very reliable Mesocolumn WordPress theme finally made an entry in a website’s fatal error logs.  I make a habit of examining website error logs, if you have a business with a web department, insist that they regularly scan and check your website error logs or subscribe to a service or has a script that can report 500 errors immediately.

Here was the error that cropped up:

[22-Apr-2016 09:27:05 UTC] PHP Fatal error: Call to undefined function get_theme_option() in /webroot/prolificfutility.com/public_html/wp-content/themes/mesocolumn-child/footer.php on line 119

500 server errors are never pleasant, it’s a fact of life if we don’t dedicate QA time to test, test and then test the sites again and again.  There is a point of diminishing returns of dedicating too much time to testing because that takes time away from web development, you know, actually creating coding solutions?!

Fixing WordPress 500 errors

This was an easy fix, because the error was easy to spot in the logs — get_theme_option function was undefined and there was a condition in the footer that called on it, so I simply wrapped the condition throwing the error first with this check:

if (function_exists('get_theme_options')) {
 // execute other code that was in the child theme here...

It’s not an ideal fix, but since I didn’t have time to investigate further, at least the page would continue loading if the get_theme_options() function was actually deprecated from the theme during a recent update.  It’s all about prioritizing your web development tasks to make sure you are optimizing your code production and in the context of other due dates, projects and considering the ROI of this task, it’s way down on the priority list.  So far down it will probably never be addressed since, truthfully, there are usually much more important and profitable web development tasks to perform.  Perhaps the next update it will fix itself, there’s no harm in doing an extra check to see if the function exists, even if they bring it back.

The more important benefit is that the page continues to load without throwing a fatal error, which is not only bad for SEO, it’s bad for the UX of every visitor and that’s always something that’s top on the priority list!