Utah’s Department of Workforce Services (DWS), in partnership with Montana’s Department of Labor, used their Workforce Innovation Fund (WIF) grant to carry out the GenLEX Initiative, a newly-developed labor exchange system that replaced the states’ existing labor exchange systems (online self-service, job matching systems).

The GenLEX initiative was designed to 1) reduce reliance on staff-assisted services and promote the use of self-service LEX; 2) provide LEX at a lower cost-per-participant; 3) address the strain and access issues at physical American Job Centers; 4) assist job seekers and students with better connection to career pathways and related education opportunities; and 5) improve Common Measures by introducing new, innovative outcomes that more accurately measure LEX success.

The evaluation of the GenLEX initiative included an impact study (which used both a randomized controlled trial (RCT) design and a quasi-experimental analysis), a descriptive analysis, and a process evaluation. The impact evaluation was conducted only in Utah because Montana did not have the capacity to maintain two labor exchange platforms simultaneously.

(The report is 188 pages long including appendices.)

Major Findings & Recommendations

The evaluation found that, overall, the new system did not result in any statistically significant changes in job seeker outcomes. Job seeker satisfaction with the system was statistically significantly lower for the new/test system compared to the standard system. In addition, employer usage generally did not change. The implementation of the GenLEX initiative was hampered by personnel changes and shifting priorities. However, after the conclusion of the WIF grant, both states continued implementing the new system, as leaders felt that the continued improvements to the system were worthwhile and would result in long-term positive changes.

The authors highlighted the following conclusions and recommendations:

  1. Several timing and pacing issues affected the implementation of the project, including the loss of important positive momentum with personnel changes early in the project, and challenges in assessing the time needed for implementation of the initiative. Evaluators noted that with other competing interests, getting the attention of staff at all levels to focus on the changes to be implemented, or to give the time and attention needed for training and skill-building around the implementation, is challenging. Regular program design interim deadlines are necessary for moving the project forward at a more consistent pace.
  2. Another group of recommendations involves the relationship-building needed with the implementation of a new initiative. Stronger relationships with senior management are needed to retain support and to stay focused on the project’s vision and goals. Engaging other partners and representatives from various areas (i.e., agency administration, IT, and financial management) is necessary to keep all involved and apprised of activities, progress, and needs.
  3. Implementing strategies for helping technology-averse staff to embrace new technology-based products and services is a necessary component of this type of initiative.
  4. Test assumptions early and often. A sample of suggested design and implementation improvements include the following: a) improved usability; b) enhanced resume management; c) improved website intuitiveness; d) provision of translation services for non-English speakers (Spanish); e) offering regular refreshers/training on the overall site; f) provision of information on Unemployment Insurance; and g) enabling worker involvement with implementation and decision making.