Monday, June 19, 2017

Org.sonarqube.ws.client.HttpException: Error 500 on Sonar

So far i was scanning small project via Sonar scanner.
For eg : the Zip file size would be 7KB, as on the below few lines.
Total lines of codes for this project was 200-400 lines.


INFO: Calculating CPD for 1 files
INFO: CPD calculation finished
INFO: Analysis report generated in 62ms, dir size=22 KB
INFO: Analysis reports compressed in 16ms, zip size=7 KB
INFO: Analysis report uploaded in 344ms

But recently i had to do a sonar scanner to a extra large project which contain 4000-5000 LOC.

So the final report file size was 15 MB.

17:55:42.494 INFO: CPD calculation finished
17:55:53.602 INFO: Analysis report generated in 9560ms, dir size=33 MB
17:56:02.134 INFO: Analysis reports compressed in 8532ms, zip size=15 MB

When sonar try to upload the 15 MB file to the soar server im getting a HttpException error.
Bellow is the full error.

17:56:02.846 ERROR: Error during SonarQube Scanner execution
org.sonarqube.ws.client.HttpException: Error 500 on http://localhost:9000/api/ce/submit?projectKey=ClinicToCloudEMR&projectName=ClinicToCloudEMR : {"errors":[{"msg":"An error has occurred. Please contact your administrator"}]}
 at org.sonarqube.ws.client.BaseResponse.failIfNotSuccessful(BaseResponse.java:36)
 at org.sonar.scanner.bootstrap.ScannerWsClient.failIfUnauthorized(ScannerWsClient.java:106)
 at org.sonar.scanner.bootstrap.ScannerWsClient.call(ScannerWsClient.java:75)
 at org.sonar.scanner.report.ReportPublisher.upload(ReportPublisher.java:177)
 at org.sonar.scanner.report.ReportPublisher.execute(ReportPublisher.java:131)
 at org.sonar.scanner.phases.PublishPhaseExecutor.publishReportJob(PublishPhaseExecutor.java:72)
 at org.sonar.scanner.phases.PublishPhaseExecutor.executeOnRoot(PublishPhaseExecutor.java:54)
 at org.sonar.scanner.phases.AbstractPhaseExecutor.execute(AbstractPhaseExecutor.java:83)
 at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:175)
 at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:143)
 at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:128)
 at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:262)
 at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:257)
 at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:247)
 at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:143)
 at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:128)
 at org.sonar.scanner.task.ScanTask.execute(ScanTask.java:47)
 at org.sonar.scanner.task.TaskContainer.doAfterStart(TaskContainer.java:86)
 at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:143)
 at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:128)
 at org.sonar.scanner.bootstrap.GlobalContainer.executeTask(GlobalContainer.java:118)
 at org.sonar.batch.bootstrapper.Batch.executeTask(Batch.java:117)
 at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:63)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
 at java.lang.reflect.Method.invoke(Unknown Source)
 at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
 at com.sun.proxy.$Proxy0.execute(Unknown Source)
 at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:233)
 at org.sonarsource.scanner.api.EmbeddedScanner.runAnalysis(EmbeddedScanner.java:151)
 at org.sonarsource.scanner.cli.Main.runAnalysis(Main.java:123)
 at org.sonarsource.scanner.cli.Main.execute(Main.java:77)
 at org.sonarsource.scanner.cli.Main.main(Main.java:61)
17:56:02.846 ERROR: 

There were no any clue on this error at the sonar log files which is inside the installation folder.

So after doing some RnDs i increase the max_allowed_packet to 512M at the Mysql My.config file.

This file is available at the C:\ProgramData\MySQL\MySQL Server 5.7.
But still im getting the same error.

So i do some changes to sonar.config file which is available at the sonar installation folder.
sonar.jdbc.url=jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&max_allowed_packet=512M 

I will attach the entire sonar.config file ,because i have done some more changes too.

Then the error went away.

# Property values can:
# - reference an environment variable, for example sonar.jdbc.url= ${env:SONAR_JDBC_URL}
# - be encrypted. See https://redirect.sonarsource.com/doc/settings-encryption.html

#--------------------------------------------------------------------------------------------------
# DATABASE
#
# IMPORTANT: the embedded H2 database is used by default. It is recommended for tests but not for
# production use. Supported databases are MySQL, Oracle, PostgreSQL and Microsoft SQLServer.

# User credentials.
# Permissions to create tables, indices and triggers must be granted to JDBC user.
# The schema must be created first.
sonar.jdbc.username=admin
sonar.jdbc.password=admin

#----- Embedded Database (default)
# H2 embedded database server listening port, defaults to 9092
#sonar.embeddedDatabase.port=9092
#----- MySQL 5.6 or greater
# Only InnoDB storage engine is supported (not myISAM).
# Only the bundled driver is supported. It can not be changed.
sonar.jdbc.url=jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&max_allowed_packet=512M


#----- Oracle 11g/12c
# - Only thin client is supported
# - Only versions 11.2.x and 12.x of Oracle JDBC driver are supported
# - The JDBC driver must be copied into the directory extensions/jdbc-driver/oracle/
# - If you need to set the schema, please refer to http://jira.sonarsource.com/browse/SONAR-5000
#sonar.jdbc.url=jdbc:oracle:thin:@localhost:1521/XE


#----- PostgreSQL 8.x/9.x
# If you don't use the schema named "public", please refer to http://jira.sonarsource.com/browse/SONAR-5000
#sonar.jdbc.url=jdbc:postgresql://localhost/sonar


#----- Microsoft SQLServer 2012/2014 and SQL Azure
# A database named sonar must exist and its collation must be case-sensitive (CS) and accent-sensitive (AS)
# Use the following connection string if you want to use integrated security with Microsoft Sql Server
# Do not set sonar.jdbc.username or sonar.jdbc.password property if you are using Integrated Security
# For Integrated Security to work, you have to download the Microsoft SQL JDBC driver package from
# http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=11774
# and copy sqljdbc_auth.dll to your path. You have to copy the 32 bit or 64 bit version of the dll
# depending upon the architecture of your server machine.
# This version of SonarQube has been tested with Microsoft SQL JDBC version 4.1
#sonar.jdbc.url=jdbc:sqlserver://localhost;databaseName=sonar;integratedSecurity=true

# Use the following connection string if you want to use SQL Auth while connecting to MS Sql Server.
# Set the sonar.jdbc.username and sonar.jdbc.password appropriately.
#sonar.jdbc.url=jdbc:sqlserver://localhost;databaseName=sonar


#----- Connection pool settings
# The maximum number of active connections that can be allocated
# at the same time, or negative for no limit.
# The recommended value is 1.2 * max sizes of HTTP pools. For example if HTTP ports are
# enabled with default sizes (50, see property sonar.web.http.maxThreads)
# then sonar.jdbc.maxActive should be 1.2 * 50 = 60.
#sonar.jdbc.maxActive=60

# The maximum number of connections that can remain idle in the
# pool, without extra ones being released, or negative for no limit.
#sonar.jdbc.maxIdle=5

# The minimum number of connections that can remain idle in the pool,
# without extra ones being created, or zero to create none.
#sonar.jdbc.minIdle=2

# The maximum number of milliseconds that the pool will wait (when there
# are no available connections) for a connection to be returned before
# throwing an exception, or <= 0 to wait indefinitely.
sonar.jdbc.maxWait=5000

#sonar.jdbc.minEvictableIdleTimeMillis=600000
#sonar.jdbc.timeBetweenEvictionRunsMillis=30000



#--------------------------------------------------------------------------------------------------
# WEB SERVER
# Web server is executed in a dedicated Java process. By default heap size is 512Mb.
# Use the following property to customize JVM options.
#    Recommendations:
#
#    The HotSpot Server VM is recommended. The property -server should be added if server mode
#    is not enabled by default on your environment:
#    http://docs.oracle.com/javase/8/docs/technotes/guides/vm/server-class.html
#
#    Startup can be long if entropy source is short of entropy. Adding
#    -Djava.security.egd=file:/dev/./urandom is an option to resolve the problem.
#    See https://wiki.apache.org/tomcat/HowTo/FasterStartUp#Entropy_Source
#
#sonar.web.javaOpts=-Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError

# Same as previous property, but allows to not repeat all other settings like -Xmx
#sonar.web.javaAdditionalOpts=

# Binding IP address. For servers with more than one IP address, this property specifies which
# address will be used for listening on the specified ports.
# By default, ports will be used on all IP addresses associated with the server.
#sonar.web.host=0.0.0.0

# Web context. When set, it must start with forward slash (for example /sonarqube).
# The default value is root context (empty value).
#sonar.web.context=
# TCP port for incoming HTTP connections. Default value is 9000.
#sonar.web.port=9000


# The maximum number of connections that the server will accept and process at any given time.
# When this number has been reached, the server will not accept any more connections until
# the number of connections falls below this value. The operating system may still accept connections
# based on the sonar.web.connections.acceptCount property. The default value is 50.
#sonar.web.http.maxThreads=50

# The minimum number of threads always kept running. The default value is 5.
#sonar.web.http.minThreads=5

# The maximum queue length for incoming connection requests when all possible request processing
# threads are in use. Any requests received when the queue is full will be refused.
# The default value is 25.
sonar.web.http.acceptCount=250

# By default users are logged out and sessions closed when server is restarted.
# If you prefer keeping user sessions open, a secret should be defined. Value is
# HS256 key encoded with base64. It must be unique for each installation of SonarQube.
# Example of command-line:
# echo -n "type_what_you_want" | openssl dgst -sha256 -hmac "key" -binary | base64
#sonar.auth.jwtBase64Hs256Secret=

# The inactivity timeout duration of user sessions, in minutes. After the configured
# period of time, the user is logged out.
# The default value is set to 3 days (4320 minutes)
# and cannot be greater than 3 months. Value must be strictly positive.
#sonar.web.sessionTimeoutInMinutes=4320


#--------------------------------------------------------------------------------------------------
# SSO AUTHENTICATION

# Enable authentication using HTTP headers
#sonar.web.sso.enable=false

# Name of the header to get the user login.
# Only alphanumeric, '.' and '@' characters are allowed
#sonar.web.sso.loginHeader=X-Forwarded-Login

# Name of the header to get the user name
#sonar.web.sso.nameHeader=X-Forwarded-Name

# Name of the header to get the user email (optional)
#sonar.web.sso.emailHeader=X-Forwarded-Email

# Name of the header to get the list of user groups, separated by comma (optional).
# If the sonar.sso.groupsHeader is set, the user will belong to those groups if groups exist in SonarQube.
# If none of the provided groups exists in SonarQube, the user won't belong to any group.
# Note that the default group will NOT be automatically added when using SSO, it should be provided in the groups list, if needed.
#sonar.web.sso.groupsHeader=X-Forwarded-Groups

# Interval used to know when to refresh name, email and groups.
# During this interval, if for instance the name of the user is changed in the header, it will only be updated after X minutes.
#sonar.web.sso.refreshIntervalInMinutes=5


#--------------------------------------------------------------------------------------------------
# COMPUTE ENGINE
# The Compute Engine is responsible for processing background tasks.
# Compute Engine is executed in a dedicated Java process. Default heap size is 512Mb.
# Use the following property to customize JVM options.
#    Recommendations:
#
#    The HotSpot Server VM is recommended. The property -server should be added if server mode
#    is not enabled by default on your environment:
#    http://docs.oracle.com/javase/8/docs/technotes/guides/vm/server-class.html
#
#sonar.ce.javaOpts=-Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError

# Same as previous property, but allows to not repeat all other settings like -Xmx
#sonar.ce.javaAdditionalOpts=
# The number of workers in the Compute Engine. Value must be greater than zero.
# By default the Compute Engine uses a single worker and therefore processes tasks one at a time.
#    Recommendations:
#
#    Using N workers will require N times as much Heap memory (see property
#    sonar.ce.javaOpts to tune heap) and produce N times as much IOs on disk, database and
#    Elasticsearch. The number of workers must suit your environment.
#sonar.ce.workerCount=1


#--------------------------------------------------------------------------------------------------
# ELASTICSEARCH
# Elasticsearch is used to facilitate fast and accurate information retrieval.
# It is executed in a dedicated Java process. Default heap size is 1Gb.

# JVM options of Elasticsearch process
#    Recommendations:
#
#    Use HotSpot Server VM. The property -server should be added if server mode
#    is not enabled by default on your environment:
#    http://docs.oracle.com/javase/8/docs/technotes/guides/vm/server-class.html
#
#sonar.search.javaOpts=-Xmx1G -Xms256m -Xss256k -Djna.nosys=true \
#  -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 \
#  -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError

# Same as previous property, but allows to not repeat all other settings like -Xmx
#sonar.search.javaAdditionalOpts=

# Elasticsearch port. Default is 9001. Use 0 to get a free port.
# As a security precaution, should be blocked by a firewall and not exposed to the Internet.
#sonar.search.port=9001

# Elasticsearch host. The search server will bind this address and the search client will connect to it.
# Default is 127.0.0.1.
# As a security precaution, should NOT be set to a publicly available address.
#sonar.search.host=127.0.0.1


#--------------------------------------------------------------------------------------------------
# UPDATE CENTER

# Update Center requires an internet connection to request https://update.sonarsource.org
# It is enabled by default.
#sonar.updatecenter.activate=true

# HTTP proxy (default none)
#http.proxyHost=
#http.proxyPort=
# HTTPS proxy (defaults are values of http.proxyHost and http.proxyPort)
#https.proxyHost=
#https.proxyPort=

# NT domain name if NTLM proxy is used
#http.auth.ntlm.domain=

# SOCKS proxy (default none)
#socksProxyHost=
#socksProxyPort=

# Proxy authentication (used for HTTP, HTTPS and SOCKS proxies)
#http.proxyUser=
#http.proxyPassword=


#--------------------------------------------------------------------------------------------------
# LOGGING

# SonarQube produces logs in 4 logs files located in the same directory (see property sonar.path.logs below),
# one per process:
#   Main process (aka. App) logs in sonar.log
#   Web Server (aka. Web) logs in web.log
#   Compute Engine (aka. CE) logs in ce.log
#   Elasticsearch (aka. ES) logs in es.log
#
# All 4 files follow the same rolling policy (see sonar.log.rollingPolicy and sonar.log.maxFiles) but it applies
# individually (eg. if sonar.log.maxFiles=4, there can be at most 4 of each files, ie. 16 files in total).
#
# All 4 files have logs in the same format:
#           1           2    3           4                       5                                                   6
# |-----------------| |---| |-|--------------------||------------------------------| |------------------------------------------------------------------------------------------------------------------------------|
# 2016.11.16 16:47:00 INFO  ce[AVht0dNXFcyiYejytc3m][o.s.s.c.t.CeWorkerCallableImpl] Executed task | project=org.sonarqube:example-java-maven | type=REPORT | id=AVht0dNXFcyiYejytc3m | submitter=admin | time=1699ms
#
# 1: timestamp. Format is YYYY.MM.DD HH:MM:SS
#    YYYY: year on 4 digits
#    MM: month on 2 digits
#    DD: day on 2 digits
#    HH: hour of day on 2 digits in 24 hours format
#    MM: minutes on 2 digits
#    SS: seconds on 2 digits
# 2: log level.
#    Possible values (in order of descending criticality): ERROR, WARN, INFO, DEBUG and TRACE
# 3: process identifier. Possible values: app (main), web (Web Server), ce (Compute Engine) and es (Elasticsearch)
# 4: SQ thread identifier. Can be empty.
#    In the Web Server, if present, it will be the HTTP request ID.
#    In the Compute Engine, if present, it will be the task ID.
# 5: logger name. Usually a class canonical name.
#    Package names are truncated to keep the whole field to 20 characters max
# 6: log payload. Content of this field does not follow any specific format, can vary in length and include line returns.
#    Some logs, however, will follow the convention to provide data in payload in the format " | key=value"
#    Especially, log of profiled pieces of code will end with " | time=XXXXms".

# Global level of logs (applies to all 4 processes).
# Supported values are INFO (default), DEBUG and TRACE
#sonar.log.level=INFO

# Level of logs of each process can be controlled individually with their respective properties.
# When specified, they overwrite the level defined at global level.
# Supported values are INFO, DEBUG and TRACE
#sonar.log.level.app=INFO
#sonar.log.level.web=INFO
#sonar.log.level.ce=INFO
#sonar.log.level.es=INFO

# Path to log files. Can be absolute or relative to installation directory.
# Default is <installation home>/logs
#sonar.path.logs=logs

# Rolling policy of log files
#    - based on time if value starts with "time:", for example by day ("time:yyyy-MM-dd")
#      or by month ("time:yyyy-MM")
#    - based on size if value starts with "size:", for example "size:10MB"
#    - disabled if value is "none".  That needs logs to be managed by an external system like logrotate.
#sonar.log.rollingPolicy=time:yyyy-MM-dd

# Maximum number of files to keep if a rolling policy is enabled.
#    - maximum value is 20 on size rolling policy
#    - unlimited on time rolling policy. Set to zero to disable old file purging.
sonar.log.maxFiles=20

# Access log is the list of all the HTTP requests received by server. If enabled, it is stored
# in the file {sonar.path.logs}/access.log. This file follows the same rolling policy as other log file
# (see sonar.log.rollingPolicy and sonar.log.maxFiles).
#sonar.web.accessLogs.enable=true

# Format of access log. It is ignored if sonar.web.accessLogs.enable=false. Possible values are:
#    - "common" is the Common Log Format, shortcut to: %h %l %u %user %date "%r" %s %b
#    - "combined" is another format widely recognized, shortcut to: %h %l %u [%t] "%r" %s %b "%i{Referer}" "%i{User-Agent}"
#    - else a custom pattern. See http://logback.qos.ch/manual/layouts.html#AccessPatternLayout.
# The login of authenticated user is not implemented with "%u" but with "%reqAttribute{LOGIN}" (since version 6.1).
# The value displayed for anonymous users is "-".
# The SonarQube's HTTP request ID can be added to the pattern with "%reqAttribute{ID}" (since version 6.2).
# If SonarQube is behind a reverse proxy, then the following value allows to display the correct remote IP address:
#sonar.web.accessLogs.pattern=%i{X-Forwarded-For} %l %u [%t] "%r" %s %b "%i{Referer}" "%i{User-Agent}" "%reqAttribute{ID}"
# Default value (which was "combined" before version 6.2) is equivalent to "combined + SQ HTTP request ID":
#sonar.web.accessLogs.pattern=%h %l %u [%t] "%r" %s %b "%i{Referer}" "%i{User-Agent}" "%reqAttribute{ID}"


#--------------------------------------------------------------------------------------------------
# OTHERS

# Delay in seconds between processing of notification queue. Default is 60 seconds.
#sonar.notifications.delay=60

# Paths to persistent data files (embedded database and search index) and temporary files.
# Can be absolute or relative to installation directory.
# Defaults are respectively <installation home>/data and <installation home>/temp
#sonar.path.data=data
#sonar.path.temp=temp


#--------------------------------------------------------------------------------------------------
# DEVELOPMENT - only for developers
# The following properties MUST NOT be used in production environments.

# Dev mode allows to reload web sources on changes and to restart server when new versions
# of plugins are deployed.
#sonar.web.dev=false

# Elasticsearch HTTP connector, for example for KOPF:
# http://lmenezes.com/elasticsearch-kopf/?location=http://localhost:9010
#sonar.search.httpPort=-1


No comments:

Post a Comment