Encoding URL with special characters

With WSO2 EI.6.1.1

When it is trying to encode url with special characters you may find an issue of values getting encoded twice.

URL to call:

http://www.mocky.io/v2/5185415ba171ea3a00704eed?fields=items[id,sku]&searchCriteria[filter_groups][0][filters][0][field]=updated_at&searchCriteria[filter_groups][0][filters][0][value]=2019-02-10 00:00:00&searchCriteria[filter_groups][0][filters][0][condition_type]=gt&searchCriteria[currentPage]=1&searchCriteria[pageSize]=5

We may use the following proxy.

<?xml version="1.0" encoding="UTF-8"?><api xmlns="http://ws.apache.org/ns/synapse" name="UrlEncoding3" context="/urlEncoding3">
<resource methods="POST">
<inSequence>
<property name="uri.var.urlcontext" value="?fields=items[id,sku]&amp;searchCriteria[filter_groups][0][filters][0][field]=updated_at&amp;searchCriteria[filter_groups][0][filters][0][value]=2019-02-10 00:00:00&amp;searchCriteria[filter_groups][0][filters][0][condition_type]=gt&amp;searchCriteria[currentPage]=1&amp;searchCriteria[pageSize]=5"/>
<log>
<property name="---beforeEncode-----" expression="get-property('uri.var.urlcontext')"/>
</log>
<property name="uri.var.httpendpointurl" expression="fn:concat('http://www.mocky.io/v2/5185415ba171ea3a00704eed', fn:url-encode($ctx:uri.var.urlcontext))"/>
<log>
<property name="---afterEncode-----" expression="get-property('uri.var.httpendpointurl')"/>
</log>
<send>
<endpoint>
<http method="POST" uri-template="{uri.var.httpendpointurl}"/>
</endpoint>
</send>
</inSequence>
<outSequence>
<respond/>
</outSequence>
<faultSequence/>
</resource>
</api>

Expected after encode:

fields=items%5Bid,sku%5D&searchCriteria%5Bfilter_groups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=updated_at&searchCriteria%5Bfilter_groups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=2019-02-10%2000:00:00&searchCriteria%5Bfilter_groups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bcondition_type%5D=gt&searchCriteria%5BcurrentPage%5D=1&searchCriteria%5BpageSize%5D=5

But in some product versions you may get it encoded twice as below.

fields=items%255Bid,sku%255D&searchCriteria%255Bfilter_groups%255D%255B0%255D%255Bfilters%255D%255B0%255D%255Bfield%255D=updated_at&searchCriteria%255Bfilter_groups%255D%255B0%255D%255Bfilters%255D%255B0%255D%255Bvalue%255D=2019-02-10%252000:00:00&searchCriteria%255Bfilter_groups%255D%255B0%255D%255Bfilters%255D%255B0%255D%255Bcondition_type%255D=gt&searchCriteria%255BcurrentPage%255D=1&searchCriteria%255BpageSize%255D=5

Therefore we can use header mediator in order to avoid this.

<api xmlns="http://ws.apache.org/ns/synapse" name="UrlEncoding" context="/urlEncoding">
<resource methods="POST">
<inSequence>
<property name="uri.var.urlcontext" value="?fields=items[id,sku]&amp;searchCriteria[filter_groups][0][filters][0][field]=updated_at&amp;searchCriteria[filter_groups][0][filters][0][value]=2019-02-10 00:00:00&amp;searchCriteria[filter_groups][0][filters][0][condition_type]=gt&amp;searchCriteria[currentPage]=1&amp;searchCriteria[pageSize]=5"/>
<log>
<property name="----beforeEncode-----" expression="get-property('uri.var.urlcontext')"/>
</log>
<property name="uri.var.httpendpointurl" expression="fn:concat('http://www.mocky.io/v2/5185415ba171ea3a00704eed', fn:url-encode($ctx:uri.var.urlcontext))"/>
<log>
<property name="---afterEncode----" expression="get-property('uri.var.httpendpointurl')"/>
</log>
<header name="To" expression="get-property('uri.var.httpendpointurl')"/>
<send>
<endpoint>
<default/>
</endpoint>
</send>
</inSequence>
<outSequence>
<respond/>
</outSequence>
<faultSequence/>
</resource>
</api>

Wire logs would be as below.

[2019-03-04 12:16:31,434] [EI-Core] DEBUG - wire HTTP-Listener I/O dispatcher-4 >> "POST /urlEncoding HTTP/1.1[\r][\n]"
[2019-03-04 12:16:31,437] [EI-Core] DEBUG - wire HTTP-Listener I/O dispatcher-4 >> "Content-Type: text/plain[\r][\n]"
[2019-03-04 12:16:31,437] [EI-Core] DEBUG - wire HTTP-Listener I/O dispatcher-4 >> "cache-control: no-cache[\r][\n]"
[2019-03-04 12:16:31,437] [EI-Core] DEBUG - wire HTTP-Listener I/O dispatcher-4 >> "Postman-Token: b700cbde-9ffa-4c92-8abb-cc72550856ef[\r][\n]"
[2019-03-04 12:16:31,437] [EI-Core] DEBUG - wire HTTP-Listener I/O dispatcher-4 >> "User-Agent: PostmanRuntime/7.6.0[\r][\n]"
[2019-03-04 12:16:31,437] [EI-Core] DEBUG - wire HTTP-Listener I/O dispatcher-4 >> "Accept: */*[\r][\n]"
[2019-03-04 12:16:31,438] [EI-Core] DEBUG - wire HTTP-Listener I/O dispatcher-4 >> "Host: 10.100.5.136:8280[\r][\n]"
[2019-03-04 12:16:31,438] [EI-Core] DEBUG - wire HTTP-Listener I/O dispatcher-4 >> "accept-encoding: gzip, deflate[\r][\n]"
[2019-03-04 12:16:31,438] [EI-Core] DEBUG - wire HTTP-Listener I/O dispatcher-4 >> "content-length: 1[\r][\n]"
[2019-03-04 12:16:31,438] [EI-Core] DEBUG - wire HTTP-Listener I/O dispatcher-4 >> "Connection: keep-alive[\r][\n]"
[2019-03-04 12:16:31,438] [EI-Core] DEBUG - wire HTTP-Listener I/O dispatcher-4 >> "[\r][\n]"
[2019-03-04 12:16:31,438] [EI-Core] DEBUG - wire HTTP-Listener I/O dispatcher-4 >> "a"
[2019-03-04 12:16:31,455] [EI-Core] INFO - LogMediator To: /urlEncoding, MessageID: urn:uuid:91070b03-60a8-4bfc-b59b-62906d2734c5, Direction: request, ----beforeEncode----- = ?fields=items[id,sku]&searchCriteria[filter_groups][0][filters][0][field]=updated_at&searchCriteria[filter_groups][0][filters][0][value]=2019-02-10 00:00:00&searchCriteria[filter_groups][0][filters][0][condition_type]=gt&searchCriteria[currentPage]=1&searchCriteria[pageSize]=5
[2019-03-04 12:16:31,457] [EI-Core] INFO - LogMediator To: /urlEncoding, MessageID: urn:uuid:91070b03-60a8-4bfc-b59b-62906d2734c5, Direction: request, ---afterEncode---- = http://www.mocky.io/v2/5185415ba171ea3a00704eed?fields=items%5Bid,sku%5D&searchCriteria%5Bfilter_groups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=updated_at&searchCriteria%5Bfilter_groups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=2019-02-10%2000:00:00&searchCriteria%5Bfilter_groups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bcondition_type%5D=gt&searchCriteria%5BcurrentPage%5D=1&searchCriteria%5BpageSize%5D=5
[2019-03-04 12:16:31,701] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-4 << "POST /v2/5185415ba171ea3a00704eed?fields=items%5Bid,sku%5D&searchCriteria%5Bfilter_groups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=updated_at&searchCriteria%5Bfilter_groups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=2019-02-10%2000:00:00&searchCriteria%5Bfilter_groups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bcondition_type%5D=gt&searchCriteria%5BcurrentPage%5D=1&searchCriteria%5BpageSize%5D=5 HTTP/1.1[\r][\n]"
[2019-03-04 12:16:31,702] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-4 << "Accept: */*[\r][\n]"
[2019-03-04 12:16:31,702] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-4 << "Postman-Token: b700cbde-9ffa-4c92-8abb-cc72550856ef[\r][\n]"
[2019-03-04 12:16:31,702] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-4 << "cache-control: no-cache[\r][\n]"
[2019-03-04 12:16:31,703] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-4 << "accept-encoding: gzip, deflate[\r][\n]"
[2019-03-04 12:16:31,703] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-4 << "Content-Type: text/plain[\r][\n]"
[2019-03-04 12:16:31,703] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-4 << "Transfer-Encoding: chunked[\r][\n]"
[2019-03-04 12:16:31,703] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-4 << "Host: www.mocky.io[\r][\n]"
[2019-03-04 12:16:31,703] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-4 << "Connection: Keep-Alive[\r][\n]"
[2019-03-04 12:16:31,703] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-4 << "User-Agent: Synapse-PT-HttpComponents-NIO[\r][\n]"
[2019-03-04 12:16:31,703] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-4 << "[\r][\n]"
[2019-03-04 12:16:31,704] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-4 << "1[\r][\n]"
[2019-03-04 12:16:31,704] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-4 << "a[\r][\n]"
[2019-03-04 12:16:31,704] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-4 << "0[\r][\n]"
[2019-03-04 12:16:31,704] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-4 << "[\r][\n]"

Written by

Senior Software Engineer — QA at WSO2

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store