uPortal 4.1.0 Release Description

Binaries linked at the bottom of this release page.

uPortal 4.1 is a "big" "minor" uPortal release including a bunch of features. This document steps through some of the changes in this release, linking to selected JIRA issues and to the relevant uPortal manual wiki pages. Much more detail is available in the under-continued-development uPortal 4.1 manual wiki space, in the full issue-tracker-generated release notes, and in the source control logs.

Security fixes

uPortal 4.1 includes several important security fixes (most of which were previously addressed in 4.0 patch releases).

These fixes are detailed in an appendix at the end of this document.

Supported Runtime Environment

uPortal 4.1 supports and runs in Java 7 and Tomcat 7, including later Tomcat 7 versions.

Feature ports to achieve Tomcat 7 compatibility include updated URL rewriting disabling configuration, and adjustment to comport with under-zealous scanning for taglib descriptors.

Respondr

uPortal 4.1 ships with a new responsive-design theme transform and skin named Respondr, complete with a login profile such that now Universality, mUniversality, and Respondr each have a login profile and Respondr is the default for all uses including for mobile devices.

Respondr includes popular Universality theme features, many of which re-implemented into layout management configuration of Regions and portlets placed into those regions.

Respondr uses Bootstrap (3) with the Paypal accessibility plugin.

mUniversality

uPortal 4.1 maintains mUniversality, with headers and footers now rendering and the JSON representation of layout now includes portlets in special areas of the page, besides just those within traditional tabs and columns.

Theme support

Respondr is in part built on better support for building themes in uPortal 4.1.

Specifically, uPortal permission checks are now available in XSL transforms (access to the Customize menu and to the Add Tab control is now permissions-based) and some JavaScript is now common between skins across themes, easing theme development and maintenance.

That said, please don't go build a whole new theme without first trying to improve Respondr to meet your needs!

Multi-tenant

uPortal 4.1 includes support for multi-tenancy with bootstrapping of tenant admin accounts via email.

Access to publish particular portlet publishing types is now gated by a SELECT_PORTLET_TYPE permission useful for tailoring the access of tenant administrators. The entire select portlet type step is skipped if a user has but one viable choice.

uPortal 4.1 tries to use permissions to determine a sensible group and category selection tree root in launching the group selection UI, addressing a use case where a tenant administrator might have permission to publish to only a portion of the user groups tree or in only a portion of the portlet category tree.

Skins in this release can be compiled and included more dynamically via the DynamicRespondrSkin portlet.

APIs

Initial progress towards a platform JSON web services API offers groups and permissions query support.

Deep linking

Deep linking is now better supported with support for tab externalId and uPortal no longer loses track of request parameters under certain login paths.

Marketplace

uPortal 4.1.0 includes a first pass at a Marketplace portlet as an alternative to the Customize Drawer for enabling discovering and selecting portlets. It supports rating portlets and displaying back to users aggregated ratings.

Marketplace is best understood as a pre-release preview of new navigation in development at the University of Wisconsin-Madison. It has significant rough edges in the initial 4.1.0 release.

Favorites

uPortal 4.1 ships with a Favorites portlet.

Favorites is best understood as a pre-release preview of new navigation in development at the University of Wisconsin-Madison. It has significant rough edges in the initial 4.1.0 release.

JPA-backed PAGS

uPortal 4.1 introduces an optional JPA-backed (in the database) configuration of PAGS (the Person Attribute Group Store) intended to obviate the XML-file-backed PAGS configuration of prior uPortal releases (and still available in this release). Both XML-backed and database-backed PAGS have a new eager regex tester available that behaves more like what you may have thought the existing regex tester did and PAGS is enhanced as regards its ability to use property values.

Notifications

A Notification Icon portlet is now included in the header with an indication of the current count of notifications.

Search

Search now features auto-complete, hides tabs if so configured, and doesn't require case matching where you wouldn't expect it to.

Layout Management

Most DLM fragments shipping in uPortal now target groups using deepMemberOf rather than (shallow) memberOf. This yields applying fragments to members of sub-groups and demonstrates the preferred practice.

The Fragment Administration portlet now saves edit permission changes properly and supports publishing links into the sidebar from a DLM fragment.

Comments in fragment-layout.xml files no longer cause parts of layouts not to be imported and the Audit DLM Fragments portlet no longer fails when the audited DLM fragment references an unrecognized portlet fname.

Import/Export

uPortal now (optionally) runs db-update when it detects this is needed.

DLM ProfileEvaluator export and import round-trip now works properly and the export flag on db-update and db-init Ant tasks is now properly honored.

Entity file archives can now be imported and values within entity exports are now consistently ordered.

JAXB importer, exporter, deleter, and upgrader provisioning failures no longer fail the entire portal initialization.

See Import-Export in the uPortal 4.1 manual wiki.

CSS, styling, and skinning

Users can now select a background image.

The DETACHED window state now provides an optional sticky header.

Styling details are improved throughout, including in Person Lookup and User and Group Permission pages. Text shadows are improved. Zoom scale problems and fixed toolbar bugs are addressed.

Tables in the uPortal UI are updated to use Datatables instead of Fluid.

Unchecked console.log usages in JavaScript are resolved.

Users and Attributes

The USER_NAME column in the uPortal database is widened to 100 characters.

Username-keyed user attribute caches are now purged on login so that attribute values will be freshly determined for the logging in user. Suggestions and example configuration for user attribute gathering are clarified, include a caching example, and the example LDAP configuration is fixed and improved.

Editing of "View user attribute" permissions is no longer broken and the password management portlet no longer fails when a user lacks edit permissions on all attributes.

The Directory portlet now warns about maximum result set size when this limits the result set and now maximizes on submit.

Person Lookup maximum results is now configurable and the result set is displayed with a scrollbar (UP-3652). cn is now included by default in directory search of LDAP (UP-3709).

The [reset-password workflow is no longer unusably bugged][UP-4054] and the portlet's behavior is otherwise improved including to use the displayName person attribute in the password reset email.

User lookup is no longer bugged and impersonation now supports selecting a target profile.

Magic permission target values are no longer misinterpreted as usernames.

Google Analytics and Event Recording

uPortal 4.1 includes Google Analytics support, improvements to the stability of the internal event recording and aggregation, and new ways to report on the internal event recording.

New ways to report on events

uPortal 4.1 includes a new Portal Activity portlet.

The statistics report no longer auto-runs on initial selection of a report in the statistics reporting portlet, allowing the opportunity to configure the report before incurring the potentially costly computation.

Reports are better (bugged reporting intervals are fixed, groups and intervals are now sorted, and column labels are better implemented) and once you've got a report you like you can now grab a permalink.

But you might prefer to do all that tracking and reporting through Google Analytics instead.

Improved internal event recording stability

In improved internal event recording stability: JpaPortalEventStore event aggregation now ignores malformed events and event aggregation now flags and avoids re-attempting raw events that it fails to process. Changing a portlet fname no longer yields a database constraint violation. Event aggregation no longer fails under MySQL.
The default behavior in absence of aggregation configuration is improved to do nothing and flushing during event aggregation is reduced. Closed aggregations on bad ordering no longer cause event processing failures and event processing now makes more of the intended effort to catch up when it falls behind.

Portal and event cache managers are merged to simplify the event aggregator's benefiting from replicated caches where implemented. uPortal 4.1 fixes a bug where AcademicTermDetails and QuarterDetails were needlessly churned.

Event processing is now database model version aware to better support updates applied via rolling restart.

Portlet Container

uPortal 4.1 picks up better JSR-286 RenderHeaders support and numerous fixes and improvements under the hood to the portlet container.

RenderHeaders support

RenderHeaders output is now included with support for two phase render. Setting Content-Type via headers now works properly.

Other Portlet Container Features

You can now flag portlets to be hidden when accessed as an impersonated user.

A magic portlet parameter allows you to specify an alternative maximized link so that maximizing a portlet can instead open the specified alternative URL (presumably, the full-fledged external application experience of whatever the portlet being maximized summarized, or a related portlet that works better to provide the maximized experience).

Caching improvements

Portlet render cache output is now purged on processAction in subsequent logins. Resource parameters are now included in cache key generation. Unused portlet cookies now purge from the database after 24 hours.

Improvements to Web-based portlet management

Access to portlet types is now filtered by requiring a new permission to select the specific portlet type. The Portlet Manager now allows administrators to filter by publication lifecycle state and now detects portlet mode support as declared in portlet.xml even when the portlet is published using a portlet type publishing workflow other than the basic "Portlet" workflow.

Portlet preferences provided via config mode are no longer lost when editing existing portlets and configuration mode now works for framework portlets and is even accessible from the portlet chrome.

Summary and Preferences steps in the Portlet Manager portlet publishing workflow are simplified and the publication workflow now prompts to go next to fragment administration upon publishing, if relevant. Portlet descriptions are now a text area instead of a too-small text box as experienced in the Portlet Manager portlet. The portlet name field now properly handles single quote characters.

Portlet Container Bugfixes

isUserInRole() is no longer bugged.

Portlet preference API default value parameters and the escapeXml portlet.xml setting are now honored.

UP_PORTLET_ENT.USER_ID is now indexed.
Hung worker tracking now works.
Commas within portlet preference values no longer inadvertently splits portlet preference values.
Granting permission to CONFIGURE a category of portlets now has the intended effect of granting permission to configure the portlets within the category.

Other bug fixes

The bugs listed here afflicted some version of uPortal prior to uPortal 4.1 and so may be a bug fix for your implementation depending on where you're upgrading from. Many of these were also fixed somewhere in the 4.0 release series. Some bugs are (instead) discussed elsewhere in these release notes where they're relevant to highlighted topics.

RDBMUserLayoutStore database resource leaks fixed
Database connection pooling now churns less by default and the attachments capability in the Simple Content Portlet now uses the proper connection pool.
Addressed broken transitive dependency through Calendar Portlet.
data-import Maven goal for Announcements no longer fails under Windows
Attempting to add a portlet to a locked tab or column now results in a sensible user-facing error.
Permission Administration asynchronous search on principal now behaves better and the portal-wide search now disables the search button on form submit to discourage launching numerous simultaneous search requests.
Delegate portlet resourceIds are now properly included in the URL.
False alarm test failures under Java 7 are addressed.
UP_JGROUPS_PING index length no longer bugs MySQL support, nor does its use of BLOB. Other keys also no longer bug MySQL for being too long.
The PortalDb DataSource is now available via JMX
Defects discovered by static code review are fixed.
Some code that had inadvertently not properly participated in transactions now participates.
GrouperEntityGroupStore is no longer afflicted with broken class cast.
The Calendar portlet default holiday feed source is now the (available) Google feed rather than a prior source that become unreliable.

Internationalization

Manage User view permissions now supports message localization.
Search and Directory Search portlet internationalization is improved.
French localization fixes and improvements.
Messages as used in JavaScript are no longer incorrectly escaped by Spring JSP tags.
Manage Portlets now uses messages more effectively.

Upgraded dependencies

Fluid Infusion upgraded to 1.5
jQuery, jQueryUI, Backbone, Underscore, and other JavaScript libraries used by Universality and mUnivesality are upgraded.
jQuery-Mobile upgraded to 1.3.2
WebProxyPortlet upgraded to 2.0.0-M2
SimpleContentPortlet upgraded to 1.0.5
Jackson to Jackson 2.
Email Preview Portlet to 2.0.3.
Notifications Portlet to 2.0.1.
NewsReaderPortlet to 3.0.5 and the New York Times education news feed is fixed.
JasigWidgetPortlets to 1.0.3
Spring to 3.1.3, which addresses a Statistics Portlet bug.
The announcements portlet is now bundled as is the calendar portlet.
CAS to a 3.5 version.

Build process

uPortal 4.1 includes improvements to the uPortal product build process.

An issue with Sonatype snapshot redirects is fixed.
Database connection settings for the Attachments portlet are no longer hard-coded and instead participate in Maven filtering.
LDAP connection settings are now in Maven filter files (UP-3623).
Now uses JDK7's native chmod if available.
The uportal-maven-plugin is now declared at the parent portlets-overlay pom rather than in individual portlet poms.
portlet data import no longer breaks portal and database initialization and portlets in the overlay can now participate in data import.
Specific orphan directories are no longer caused by the full build due to CAS goofiness.
The build process usage of Ant is improved through increased use of PortalShellBuildHelper.
WebProxyPortlet's use of datasource.properties now works more like you'd expect.
Old, broken, unneeded Ant targets are removed.
Quickstart scripts now wait for Tomcat / HSQL.
dbtest now returns.
md5passwd is no longer broken.
The compiled CSS results of SASS compilation are no longer included in the source code, adhering to the principle that artifacts compiled from source should not themselves be in the source.
Environment filters now apply more widely and include email properties.
Optionally, the build process can incorporate some configuration from outside the uPortal source directory.
The uportal-maven-plugin now properly copies MANIFEST.MF files within war files handled by deploy-ear.
uPortal now benefits from basic Travis-CI continuous integration such that commits to and proposed pull request merges to the source code on GitHub are automatically compiled and unit tested with feedback to uPortal developers on detected failures. This provides an additional safety net in uPortal open source software product development and it also provides a starting point for Travis-CI integration for institution-local uPortal projects using GitHub.
JpaClusterLockDaoTest testConcurrentCreateLocking() no longer intermittently false-fails the build.

Clustering

jGroups peer discovery is improved and uPortal 4.1 includes configuration allowing CAS Clearpass to work in clustered environments.

Logging

uPortal 4.1 adopts slf4j and Logback with streamlined logging initialization. JCL, Log4j, and java.util.logging shims redirect other logging frameworks through slf4j. Usernames are now included in thread names where relevant.

uPortal now includes an exception logging filter on all requests with additional diagnostic information.

The bundled Calendar portlet now includes logging configuration and portlet overlays now include logging configuration.

Ease of ClearPass implementation

uPortal 4.1.0 includes example ClearPass configuration for use of ClearPass in a clustered uPortal environment.
(UP-4108) and improves the underlying uPortal side of the ClearPass integration (UP-3621). The key for encrypting passwords in memory is now set in portal.properties (you did set this to something other than the default in your implementation, right?).

Appendix: Security fixes in uPortal 4.1

Enforces IMPERSONATE permissions

Whereas under uPortal 4.0.8 users with access to User Administration could impersonate any user regardless of IMPERSONATE permissions, in uPortal 4.1 (and in uPortal 4.0.9) IMPERSONATE permission requirements are enforced properly.

Enforces CONFIG permissions on portlet publications

uPortal 4.0 releases prior to uPortal 4.0.13.1 did not properly require CONFIG permission to edit portlet configuration. uPortal 4.1 fixes this.

Enforces MANAGE permissions on portlet publications

uPortal 4.0 releases prior to 4.0.13.1 did not properly require MANAGE permission to manage portlet registrations. uPortal 4.1 fixes this.

Blocks a markup injection issue in usernames

uPortal 4.1.0 blocks a way to create users through the administrative UI with usernames that amount to a markup injection attack.

Blocks illicit proxy authentication into uPortal

uPortal 4.0.11 shipped with configuration of the CAS Client filter over uPortal such that it would accept any proxy ticket, whereas typically uPortal itself out not to accept any proxy tickets. This issue was addressed in uPortal 4.0.11.1 and later, including in uPortal 4.1.0.

Removes `acceptAnyProxy` in included CAS server configuration

uPortal 4.0.14 shipped with configuration of the included CAS server such that if ClearPass were implemented and the configuration were not properly updated, the CAS ClearPass component would accept any proxy ticket rather than only those from specifically authorized services, which could expose end user credentials to illicit access by other CAS-using applications. uPortal 4.1.0 corrects this (as does current 4.0-patches).

Blocks tag injection in portlet titles

uPortal 4.1 blocks injection of tags in portlet titles at portlet registration.

Thanks to

These participants contributed commits to this release:

Special thanks to MyUW team student employee Ahad Zaman for additional testing of the uPortal 4.1.0 release.

uPortal 4.1.0