Update to v0.4

README.md

@@ -1,7 +1,7 @@
 Typo3-Enumerator
 ===============
 
-Typo3-Enumerator is an open source penetration testing tool that automates the process of detecting the [Typo3](https://typo3.org) CMS and its installed [extensions](https://typo3.org/extensions/repository/?id=23&L=0&q=&tx_solr[filter][outdated]=outdated%3AshowOutdated) (also the outdated ones!).
+Typo3-Enumerator is an open source penetration testing tool that automates the process of detecting the [Typo3](https://typo3.org) CMS and it's installed [extensions](https://typo3.org/extensions/repository/?id=23&L=0&q=&tx_solr[filter][outdated]=outdated%3AshowOutdated) (also the outdated ones).
 If the --top parameter is set to a value, only the specified most downloaded extensions are tested.
 
 It is possible to do all requests through the [TOR Hidden Service](https://www.torproject.org/) network or [Privoxy](http://sourceforge.net/projects/ijbswa/files/). Also you can combine TOR with Privoxy in order to prevent DNS leakage.
@@ -15,20 +15,18 @@ Preferably, you can download Type-Enumerator by cloning the [Git](https://github
 
     git clone https://github.com/whoot/Typo-Enumerator.git
 
-Typo-Enumerator works with [Python](http://www.python.org/download/) version **2.6.x** and **2.7.x** on Debian/Ubuntu, RedHat and Windows platforms.
+Typo-Enumerator works with [Python](http://www.python.org/download/) version **3.x** on Debian/Ubuntu, RedHat and Windows platforms.
 
 On Windows you might need to install following packages:
 
-* [Requests](https://pypi.python.org/pypi/requests/2.3.0)
+* [Requests](https://pypi.python.org/pypi/requests/)
+* [Colorama](https://pypi.python.org/pypi/colorama)
 
 On Redhat you can install all needed packages with easy_install:
 
 	easy_install argparse
 	easy_install requests
-
+	easy_install colorama
-To make it look pretty, you may want to install colorama:
-
-* [Colorama](https://pypi.python.org/pypi/colorama)
 
 If you want to use Typo-Enumerator with TOR, you need the [SocksiPy](http://socksipy.sourceforge.net/) module.
 On Debian/Ubuntu you can install it with apt-get:
@@ -65,4 +63,22 @@ Links
 * Download: [.tar.gz](https://github.com/whoot/Typo-Enumerator/tarball/master) or [.zip](https://github.com/whoot/Typo-Enumerator/archive/master)
 * Changelog: [Here](https://github.com/whoot/Typo-Enumerator/blob/master/doc/CHANGELOG.md)
 * TODO: [Here](https://github.com/whoot/Typo-Enumerator/blob/master/doc/TODO.md)
-* Issue tracker: https://github.com/whoot/Typo-Enumerator/issues
+* Issue tracker: https://github.com/whoot/Typo-Enumerator/issues
+
+# License
+
+Typo3 Enumerator - Automatic Typo3 Enumeration Tool
+Copyright (c) 2015 Jan Rude
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see [http://www.gnu.org/licenses/](http://www.gnu.org/licenses/)

doc/CHANGELOG.md

@@ -1,3 +1,11 @@
+## Version 0.4
+
+* Using Python 3.x now!
+* Using classes and objects
+* Better searching algorithm
+* Better threading (fixed local network DOS when cheking a lot of extensions)
+* Clearer output
+
 ## Version 0.3.3
 
 * Extensions are now saved into different files, separated by state (experimental | alpha | beta | stable | outdated | all). This makes it possible to check more specific ones.

doc/TODO.md

@@ -1,9 +1,4 @@
 # TODO
 
-* Find better searching algorithm for Typo3 login.
-* Search for Typo3 version-specific extensions
-* Some extensions don't have any version information. These extensions must be listed in settings.NO_VERSIONINFO.
-* Maybe use one library for all requests
 * Add screenshot
-
+* Documentation
-* verbose gruscht gscheid behandeln

extensions/experimental_extensions

@@ -0,0 +1,173 @@
+alternative_rte
+kh_photoweb
+rte_mswordclean
+frontendformslib
+mmforeign
+cc_cbrowse
+articles
+sl_css_imgtext
+google_api_search
+formbuilder
+dam_demo
+nicosdirectory
+em2_test
+xml_contentrendering
+xajax_tutor
+csevents
+lz_chess
+convertutf
+beuser_tracking
+mz_miniworkflow
+xajax_example
+pdf_export
+cite
+jp_events
+sk_svg
+ilis_newsticker_doc
+bm_basics
+cwt_hcl
+ahxol
+rsp_webmail
+ameos_demo_formidable
+svo_tvplaintext
+eu_sso_horde
+p2_menudom
+zor_ts2str
+aofeusersystem
+test_contux
+ggt_catwalk_template
+cp_validator
+cc_tsquicklink
+hsr_charts
+hjm_singlesignon
+mpg_staff
+link_by_pageid
+fl_googleadsense
+sici_tmplonfs
+masi_utf8fe36
+jg_wishlist
+ameos_demoformidable050
+kb_test_em
+skraemer_tmplsel
+doc_formbuilder_fr
+openimmoman
+jm_recaptcha_example
+programm_roadmap
+wag_ntlm_sso
+pr_mailcampaign
+rhu_excelexplorer
+felib
+eim2mvc
+cherries
+sm_charsethelper
+mvwa_fortune
+ws_test
+survey
+masi_utf8fs
+rhu_member
+dsn_sopform
+ggt_catwalk_wardrobe
+rg_yfm
+dsn_sopdisplay
+cps_eventsnow
+test_wj
+feusers2xml
+svq_feuser_filemanager
+ch_mandelbrot
+subdoktypes
+dr_directors_list
+ck_heim
+sm_test123
+tvp_clipboard
+kickstarter__mvc_ex
+joytopia
+tvp_newcewizard
+abcstarter
+tw_shop
+vegb2cmnt
+party
+ms_eidtest
+extfileupload
+svq_fe_user_mailform
+efs
+test_uploaddependency
+jhe_dam_extender
+dbreplace
+spriteiconoverview
+bb_easyforms
+abcconfig
+ms_fluid
+eventmanagement
+ajax_report
+smu_chc_ext
+ch_flash_carrousel
+tcaobjects_demo
+jr_webmail
+wsefs
+rhu_csvimport
+pb_rsslaufschrift
+ch_bramacroofsimulator
+european
+p2_langfix_42
+clanbase
+ter_tests
+meta_openoffice
+st_validation_lpl
+rhu_events
+t3info
+ch_bramacproducts
+sort_table
+maja_condrequired
+alumnos
+organizacionacademica
+bonus
+hh_multipageform_example
+lz_lp_dm_log_fe
+dsxsyndication
+ba_company
+zitatdt
+svq_ebay
+rm_staticfile
+automator
+contactformgenerator
+rg_links
+audio_conversion
+error
+wow_raid
+mbbrowserid
+mf_trainmanagement
+rg_usuarios
+rg_patrocinio
+sp_newsteaserbox_hookexample
+redirectlog
+party_tests
+rb_tf_database
+asvtiger
+belink_syslang
+buildtools
+rg_empresas
+tc_fbconnect
+rf_library
+treppenpfosten_katalog
+ffunews
+dre_besearch
+elnews
+moox_template_free017
+tagger
+dbal_utility
+ft3_empty
+og_base
+ecs_steam
+moox_news_twitter
+femanagerextended
+ter_upload_test
+simplemvc_helloworld
+lo_backendhelper
+tgm_kickstart
+downloads
+visitorlist
+ckeditor
+ctefan_test
+moox_news_geoinfo
+jh_extstatus
+jh_pwcomments_plugin

extensions/outdated_extensions

@@ -0,0 +1,298 @@
+newloginbox
+ter_update_check
+csh_de
+csh_nl
+th_mailformplus
+sr_static_info
+date2cal
+fh_library
+kb_md5fepw
+rte_pb_htmlarea
+maag_randomimage
+kj_recycler
+news_plus
+css_styled_imgtext
+csh_fr
+dam_file
+danp_documentdirs
+a1_ttnews
+sr_rtehtmlarea_xpblue
+eco_gal
+dmmjobcontrol
+rte_conf
+smile_workflow
+typo3_tut
+modern_skin
+csh_ru
+fancycorners
+wt_directory
+newloginbox_tmplable
+bzd_staff_directory
+cc_devlog
+sr_language_detect
+multicolumn
+rte_chooser
+sp_news_catimgs
+doc_core_tsref
+dkd_newsmulticats
+csh_dk
+kb_unpack
+realurlsettings
+beuser_ip_lock
+csh_es
+addressgroups
+fl_staticfilecache
+cc_metaexif
+sr_rtehtmlarea_bluelook
+jf_headerslide
+doc_core_tsconfig
+imailframe
+kb_cont_slide
+cc_metaexec
+csh_it
+news_pack
+pk_limitmenutolang
+df_dmailer
+doc_core_tstemplates
+pt_payment
+csh_jp
+livestatframe
+fed
+wmdb_contentwizard
+doc_core_inside
+smile_workflowdoc
+csh_pl
+ah_flashinnews
+xtemplate
+mh_auto_alias
+gb_workflow
+ameos_feuser_manager
+tableswithoutp
+doc_core_cgl
+sg_zfeeditlib
+aau_pbsurvey
+joboffers_feedit
+gforms
+kb_shop
+danp_md5fepassword
+doc_core_ts
+cal_tt_news_service
+rrzn_pagelinks
+me_templavoilalayout2
+gb_feuserregistratio
+av_weblinks
+doc_core_api
+goof_fotoboek_fix1
+maag_formcaptcha
+doc_tut_quickstart
+sg_feautologin
+lz_https
+cc_infotablesmgm
+rtp_smarty
+csh_cz
+partner_fe
+patch1822
+csh_ch
+csh_ar
+mw_macmenu
+doc_core_services
+sg_feautologinnest
+csh_kr
+news_displaysingle
+cal_news_event_service
+smile_categorization
+framed_header
+efafontsize
+doc_inst_upgr
+df_feuser_register_ext
+fl_ebayinfo
+sr_readmail_analyse
+googleanalytics
+csh_tr
+csh_hu
+onet_ttprodoffers
+danp_extrdfexport
+df_mailformplus_ext
+csh_se
+sg_fenewsedit
+csh_sk
+cobwebphpadsnew
+dynbeedit
+csh_gr
+glossarysearch
+csh_hk
+csh_br
+dubletfinder
+prototypejs
+hsapp_longerfeusername
+wa_contentrenderinghook
+de_contentorganizer
+danp_skinsupport
+alt_forms_field_title
+danp_webcatalog
+csh_si
+abz_labels
+gp_404page
+csh_bg
+csh_ua
+formidabledatetime
+mh_multimedia_ext
+sav_library
+eco_cal
+stucki_cache_imagesizes
+perfectlightboxjquery
+csh_pt
+gt_typo3_localization
+csh_hr
+csh_ro
+tmpl_ice_3columns
+csh_fi
+csh_no
+mhnotifychanger
+doc_ephp_install_fr
+csh_ca
+kb_renmultp
+cron_autoaccesskeys
+maob_mmcache
+cc_meta_xmp
+csh_th
+wildside_extbase
+extendcobj
+essayeca_cal
+sms_directshortcuts
+tm_dl
+csh_ba
+csh_he
+csh_lt
+salutationswitcher
+csh_is
+sg_adradd
+xds_sermon_base
+csh_gl
+csh_lv
+csh_et
+phpbb3_auth
+doc_l10nguide
+html5meta_t3lib_pagerenderer
+twittersearch
+tex_script
+donations
+sp_betterflex
+localphpinclude
+tm_classes
+danp_userlisttemplate
+cobweb_protector
+tebay
+yag_theme_perfectlightbox
+rtehtmlarea_definitionlist
+eco_content
+softwarecenter
+csh_vn
+tm_minijoboffers
+paysuite
+idaa_fe_utilies
+go_maps_ap
+mailformplusplus
+ak_mobile_device
+iwbase
+eu_correcturls
+rtehtmlarea_dummyplugin
+ws_imageeffects
+jq_lightbox2
+ad_google_maps_api
+kk_mailformpluslist
+yag_theme_fancybox
+maag_sendmail
+xds_sermon_player
+powermail_static_template
+egovapi
+ts45min_de
+t3blogjquery
+cl_jquery
+extensionlist
+fe_db_browser
+mm_forum_comments
+ab_swiftmailer
+ch_hidedefaultlang
+larsp_fussballde_js
+stfl_startendtime
+completebackup
+speedy
+sav_library_extends
+mm_forum_news
+flow4t3
+browser_tut_ajax_en
+sav_library_mvc
+larsp_tagcloud
+yiid_like
+tm_qsdb
+mpm
+templatelib
+doc_guide_install
+cacheexpire
+facebook
+mpr
+displaycontroller_advanced
+smile_form_archive
+tagpackprovider
+dfluess
+doc_core_tca
+redirection
+jhe_adventcalender
+sav_library_example5
+xliff
+maag_imagerotator
+metadata_ts
+remote_server
+doc_tut_ts45
+datadisplay
+form4_doktypes
+st_readmore
+mak_randlistnum
+extended_sys_note
+static_info_tables_ga
+advancedform
+delete_staticfile_by_3party
+ics_errorhandler
+ods_workspace_mail
+tm_gallery
+extend_dcdgooglemap
+ttnews_href_marker
+sav_library_mvc_example0
+doc_tut_editors
+st_metatags
+ics_templavoila_mirgation_tool
+doc_core_skinning
+doc_guide_security
+ttnewscacheexpire
+form4_contentpagination
+realurl_autoconf_autodelete
+paymentlib_dibs
+paymentlib_quickpay_dk
+smile_jumpurl_fix
+tgm_gallery
+tm_cssfilelinks
+tsincludeorder
+tgmv_gallery
+tm_import
+nc_videostatistics
+displaycontroller_debug
+st_gfi_taleo
+dialogcentral
+dscentral
+jb_metaexec_doc
+maag_cenoshop
+view
+attachmentdelete
+form4_filecache
+coo_facebook
+filedeletion
+uploadtest
+form4_faq
+browser_tut_map_en
+mm_forum_blog
+external_link_parameter
+wt_spamshield_formhandler
+coreupdate
+form4_pages_counter
+fluidcontent_fed

lib/__init__.py

@@ -1,3 +0,0 @@
-#!/usr/bin/env python
-
-pass

lib/check_installation.py

@@ -0,0 +1,83 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+#-------------------------------------------------------------------------------
+# Typo3 Enumerator - Automatic Typo3 Enumeration Tool
+# Copyright (c) 2015 Jan Rude
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/](http://www.gnu.org/licenses/)
+#-------------------------------------------------------------------------------
+
+import re
+import sys
+from colorama import Fore
+from lib.request import Request
+from lib.output import Output
+
+class Typo3_Installation:
+	"""
+	This class checks, if Typo3 is used on the domain.
+	If Typo3 is used, a link to the login page is shown.
+
+		name: 		URL of the domain
+		extensions:	Extensions to check for
+		typo3:		If Typo3 is installed
+	"""
+	@staticmethod
+	def run(domain):
+		login_found = Typo3_Installation.search_login(domain)
+		if not login_found:
+			Typo3_Installation.check(domain)
+
+	# Searching for Typo3 references in HTML comments
+	@staticmethod
+	def check(domain):
+		response = Request.get_request(domain.get_name(), '/')
+		try:
+			print(Fore.GREEN + '[!] fe_typo_user:'.ljust(32) + response[2].cookies['fe_typo_user'] + Fore.RESET)
+			domain.set_typo3()
+		except:
+			try:
+				regex = re.compile('[Tt][Yy][Pp][Oo]3 (\d{1,2}\.\d{1,2}\.[0-9][0-9]?)')
+				searchVersion = regex.search(response[0])
+				version = searchVersion.groups()
+				domain.set_typo3()
+				domain.set_typo3_version(version[0].split()[0])
+				return True
+			except:
+				try:
+					regex = re.compile('TYPO3(.*)', re.IGNORECASE)
+					searchHTML = regex.search(response[0])
+					searchHTML.groups()[0]
+					domain.set_typo3()
+					return True
+				except:
+					return False
+
+
+	# Searching Typo3 login page
+	@staticmethod
+	def search_login(domain):
+		response = Request.get_request(domain.get_name(), '/typo3/index.php')
+		Request.interesting_headers(response[1])
+		try:
+			regex = re.compile('<title>(.*)</title>', re.IGNORECASE)
+			searchTitle = regex.search(response[0])
+			title = searchTitle.groups()[0]
+			if 'TYPO3' in title or 'TYPO3 SVN ID:' in response[0]:
+				domain.set_typo3()
+				domain.set_login_found()
+				return True
+		except:
+			pass
+		return False

lib/domain.py

@@ -0,0 +1,83 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+#-------------------------------------------------------------------------------
+# Typo3 Enumerator - Automatic Typo3 Enumeration Tool
+# Copyright (c) 2015 Jan Rude
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/](http://www.gnu.org/licenses/)
+#-------------------------------------------------------------------------------
+
+class Domain(object):
+	"""
+	This class stores following information about a domain:
+		name: 					URL of the domain
+		typo3:					If Typo3 is installed
+		typo3_version:			Typo3 Version
+		login_found:			determines of the default login page was found or not
+		extensions:				List of extensions to check for
+		installed_extensions:	List of all installed extensions
+	"""
+	def __init__(self, name, ext_state, top=False):
+		if not ('http' in name):
+			self.__name = 'http://' + name
+		else:
+			self.__name = name
+		self.__typo3 = False
+		self.__typo3_version = ''
+		self.__login_found = False
+		self.__extension_config = [ext_state, top]
+		self.__extensions = None
+		self.__installed_extensions = {}
+
+	def get_name(self):
+		return self.__name
+
+	def set_name(self, name):
+		self.__name = name
+
+	def get_extensions(self):
+		return self.__extensions
+
+	def set_extensions(self, extensions):
+		self.__extensions = extensions
+
+	def get_extension_config(self):
+		return self.__extension_config
+
+	def get_installed_extensions(self):
+		return self.__installed_extensions
+
+	def set_installed_extensions(self, extension):
+		self.__installed_extensions[extension] = False
+
+	def set_installed_extensions_version(self, extension, ChangeLog):
+		self.__installed_extensions[extension] = ChangeLog
+
+	def get_typo3(self):
+		return self.__typo3
+
+	def set_typo3(self):
+		self.__typo3 = True
+
+	def set_typo3_version(self, version):
+		self.__typo3_version = version
+
+	def get_typo3_version(self):
+		return self.__typo3_version
+
+	def get_login_found(self):
+		return self.__login_found
+
+	def set_login_found(self):
+		self.__login_found = True

lib/extensions.py

@@ -1,113 +1,77 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+#-------------------------------------------------------------------------------
+# Typo3 Enumerator - Automatic Typo3 Enumeration Tool
+# Copyright (c) 2015 Jan Rude
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/](http://www.gnu.org/licenses/)
+#-------------------------------------------------------------------------------
 
-"""
+import os.path
-Copyright (c) 2014 Jan Rude
+from lib.request import Request
-"""
+from lib.output import Output
+from lib.thread_pool import ThreadPool
 
-import re
+class Extensions:
-import os
+	def __init__(self, ext_state, top):
-import sys
+		self.__ext_state = ext_state
-import time
+		self.__top = top
-import urllib2
-from Queue import Queue
-try:
-	from colorama import Fore
-except:
-	pass
-from os.path import isfile
-from threading import Thread, Lock
-from lib import settings
 
-def generate_list():
+	def load_extensions(self):
-	print ''
+		extensions = []
-	for ext_file in settings.EXTENSION_FILE:
+		for state in self.__ext_state:
-		if not isfile(os.path.join('extensions', ext_file)):
+			ext_file = state + '_extensions'
-			output("Could not find extension file " + ext_file 
+			if not os.path.isfile(os.path.join('extensions', ext_file)):
-					+ "\nPossible values are: experimental | alpha | beta | stable | outdated | all", False)
+				raise Exception("\n\nCould not find extension file " + ext_file + '!\nTry --update')
-			sys.exit(-1)
 
-		with open(os.path.join('extensions', ext_file), 'r') as f:
+			with open(os.path.join('extensions', ext_file), 'r') as f:
-			count = 0
+				count = 0
-			print "[+] Loading:", ext_file
+				for extension in f:
-			for extension in f:
+					if not(self.__top is None):
-				if settings.TOP_EXTENSION > count:
+						if count < self.__top:
-					settings.EXTENSION_LIST.append(extension.split('\n')[0])
+							extensions.append(extension.split('\n')[0])
-					count += 1
+							count += 1
-				else:
-					f.close()
-					return
-
-def copy():
-	for extension in settings.EXTENSION_LIST:
-		settings.in_queue.put(extension)
-
-# Searching installed extensions
-# Check version if getting 200 or 403.
-def check_extension():
-	while True:
-		extension = settings.in_queue.get()
-		for path in settings.EXTENSION_PATHS:
-			try:
-				req = urllib2.Request(settings.DOMAIN + path + extension + '/', None, settings.user_agent)
-				connection = urllib2.urlopen(req, timeout = settings.TIMEOUT)
-				connection.close()
-				check_extension_version(path, extension)
-				settings.in_queue.task_done()
-				return
-			except urllib2.HTTPError, e:
-				if e.code == 403:
-					check_extension_version(path, extension)
-					settings.in_queue.task_done()
-					return
-			except urllib2.URLError, e:
-				pass
-				#retry = raw_input('Error on checking ' + extension + ': ' + str(e.reason) + '\nRetrying? (y/n) ')
-				#if retry is 'y':
-				#	settings.in_queue.put(extension)
-		# if extension is not in any given path, it's not installed
-		if settings.verbose:
-			output(extension.ljust(32) + 'not installed', False)
-		settings.in_queue.task_done()
-
-# Searching version of installed extension
-def check_extension_version(path, extension):
-	settings.EXTENSIONS_FOUND += 1
-	# if no version information is available, skip version search
-	if extension in settings.NO_VERSIONINFO:
-		if settings.verbose:
-			output(extension.ljust(32) + 'installed (no version information available)', True)
-		else:
-			output(extension.ljust(32) + 'installed', True)
-	else:
-		try:
-			request = urllib2.Request(settings.DOMAIN + path + extension +'/ChangeLog', None, settings.user_agent)
-			response = urllib2.urlopen(request, timeout = settings.TIMEOUT)
-			changelog = response.read(1500)
-			response.close()
-			try:
-				regex = re.compile("(\d{1,2}\.\d{1,2}\.?[0-9]?[0-9]?[' '\n])")
-				searchVersion = regex.search(changelog)
-				version = searchVersion.groups()
-				output(extension.ljust(32) + 'installed (v' + version[0].split()[0] + ')', True)
-			except:
-				try:
-					regex = re.compile("(\d{2,4}[\.\-]\d{1,2}[\.\-]\d{1,4})")
-					search = regex.search(changelog)
-					version = search.groups()
-					output(extension.ljust(32) + 'installed (last entry from ' + version[0] + ')', True)
-				except:
-					if settings.verbose:
-						output(extension.ljust(32) + 'installed (no version information found)', True)
 					else:
-						output(extension.ljust(32) + 'installed', True)
+						extensions.append(extension.split('\n')[0])
-		except:
+				f.close()
-			output(extension.ljust(32) + "installed", True)
+		return extensions
 
-def output(message, status):
+	def search_extension(self, domain, extensions):
-	if settings.COLORAMA:
+		thread_pool = ThreadPool()
-		if status:
+		for ext in extensions:
-			print Fore.GREEN + message + Fore.RESET
+			# search local installation path
-		else:
+			thread_pool.add_job((Request.head_request, (domain.get_name(), '/typo3conf/ext/' + ext)))
-			print Fore.RED + message + Fore.RESET
+			# search global installation path
-	else:
+			thread_pool.add_job((Request.head_request, (domain.get_name(), '/typo3/ext/' + ext)))
-		print message
+		thread_pool.start(6)
+
+		for installed_extension in thread_pool.get_result():
+			domain.set_installed_extensions(installed_extension[1][1])
+
+	def search_ext_version(self, domain, extension_dict):
+		thread_pool = ThreadPool()
+		for extension_path in extension_dict:
+			thread_pool.add_job((Request.head_request, (domain.get_name(), extension_path + '/ChangeLog')))
+			thread_pool.add_job((Request.head_request, (domain.get_name(), extension_path + '/Readme.txt')))
+		
+		thread_pool.start(6, True)
+
+		for changelog_path in thread_pool.get_result():
+			ext, path = self.parse_extension(changelog_path)
+			domain.set_installed_extensions_version(path, ext[4])
+
+	def parse_extension(self, path):
+		ext = (path[1][1]).split('/')
+		path = ext[0] + '/' + ext[1] + '/' + ext[2] + '/' + ext[3]
+		return (ext, path)

lib/login.py

@@ -1,120 +0,0 @@
-#!/usr/bin/env python
-
-"""
-Copyright (c) 2014 Jan Rude
-"""
-
-import re
-import sys
-import requests
-import urllib2
-from colorama import Fore
-from lib import settings
-if sys.platform.startswith('win32') or sys.platform.startswith('cygwin'):
-	import msvcrt
-
-# Searching Typo3 login page
-def search_login():
-	try:
-		r = requests.get(settings.DOMAIN + '/typo3/index.php', allow_redirects=False, timeout=settings.TIMEOUT, headers=settings.user_agent, verify=False)
-		statusCode = r.status_code
-		httpResponse = r.text
-		if statusCode == 200:
-			return check_title(httpResponse, r.url)
-		elif (statusCode == 301) or (statusCode == 302):
-			location = r.headers['location']
-			answer = ''
-			if sys.platform.startswith('win32') or sys.platform.startswith('cygwin'):
-				print 'Got redirect to: ' + str(location) + '\nFollow? (y/n) '
-				answer = msvcrt.getche()
-			elif sys.platform.startswith('linux'):
-				answer = raw_input('Got redirect to: ' + str(location) + '\nFollow? (y/n) ')
-			if answer is 'y':
-				locsplit = location.split('/')
-				settings.DOMAIN = locsplit[0] + '//' + locsplit[2]
-				return 'redirect'
-			else:
-				return check_title(httpResponse, r.url)
-		elif statusCode == 404:
-			return False
-		else:
-			print 'Oops! Got unhandled code:'.ljust(32) + str(statusCode) + ': ' + str(r.raise_for_status())
-	except requests.exceptions.Timeout:
-		output('Connection timed out')
-	except requests.exceptions.TooManyRedirects:
-		output('Too many redirects')
-	except requests.exceptions.RequestException as e:
-		output(str(e))
-
-# Searching for Typo3 references in title
-def check_title(response, url):
-	try:
-		regex = re.compile('<title>(.*)</title>', re.IGNORECASE)
-		searchTitle = regex.search(response)
-		title = searchTitle.groups()[0]
-		if 'TYPO3' in title or 'TYPO3 SVN ID:' in response:
-			print 'Typo3 Login:'.ljust(32) + Fore.GREEN + url + Fore.RESET
-			return True
-	except:
-		pass
-	return False
-
-# Searching for Typo3 references in HTML comments
-def check_main_page():
-	req = urllib2.Request(settings.DOMAIN, None, settings.user_agent)
-	req.add_header('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8')
-	try:
-		connection = urllib2.urlopen(req, timeout = settings.TIMEOUT)
-		response = connection.read()
-		connection.close()
-		try:
-			cookie = connection.info().getheader('Set-Cookie')
-			if 'fe_typo_user' in cookie:
-				return bad_url()
-		except KeyboardInterrupt:
-			output('\nReceived keyboard interrupt.\nQuitting...')
-			exit(-1)
-		except:
-			try:
-				regex = re.compile('TYPO3(.*)', re.IGNORECASE)
-				searchHTML = regex.search(response)
-				searchHTML.groups()[0]
-				try:
-					regex = re.compile('[Tt][Yy][Pp][Oo]3 (\d{1,2}\.\d{1,2}\.[0-9][0-9]?[' '\n])')
-					searchVersion = regex.search(response)
-					version = searchVersion.groups()
-					settings.TYPO_VERSION = 'Typo3 ' + version[0].split()[0]
-				except:
-					pass
-				return bad_url()
-			except:
-				pass
-	except Exception, e:
-		if '404' in str(e):
-			output(str(e) + '\nPlease ensure you entered the right url')
-		else:
-			output(str(e))
-		return 'skip'
-	return False
-
-def bad_url():
-	print 'Typo3 Login:'.ljust(32) + 'Typo3 is used, but could not find login'
-	print ''.ljust(32) + 'This could result in \'no extensions are installed\'.'
-	print ''.ljust(32) + 'Seems like something is wrong with the given url.'
-	answer = ''
-	if sys.platform.startswith('win32') or sys.platform.startswith('cygwin'):
-		print ''.ljust(32) + 'Try anyway (y/n)? '
-		answer = msvcrt.getch()
-	elif sys.platform.startswith('linux'):
-		answer = raw_input(''.ljust(32) + 'Try anyway (y/n)? ')
-	if answer is 'y':
-		return True
-	return 'skip'
-
-# printing error messages
-def output(message):
-	if settings.COLORAMA:
-		print Fore.RED
-	print message
-	if settings.COLORAMA:
-		print Fore.RESET

lib/output.py

@@ -1,22 +1,49 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+#-------------------------------------------------------------------------------
+# Typo3 Enumerator - Automatic Typo3 Enumeration Tool
+# Copyright (c) 2015 Jan Rude
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/](http://www.gnu.org/licenses/)
+#-------------------------------------------------------------------------------
 
-"""
-Copyright (c) 2014 Jan Rude
-"""
-
-from Queue import Queue
 from colorama import Fore
-from threading import Thread, Lock
-from lib import settings
 
-# Output thread
+class Output:
-def thread():
+	"""
-	while settings.out_queue is not settings.out_queue.empty():
+	This class handles the output
-		try:
+	"""
-			extension = settings.out_queue.get()
+	def __init__(self):
-			if not "not installed" in extension:
+		pass
-				settings.EXTENSIONS_FOUND += 1
+
-			print(extension)
+	def typo3_installation(domain):
-			settings.out_queue.task_done()
+		print('')
-		except Exception, e:
+		print('[+] Typo3 default login:'.ljust(30) + Fore.GREEN + domain.get_name() + '/typo3/index.php' + Fore.RESET)
-			print "Oops! Got:", e
+		print('[+] Typo3 version:'.ljust(30) + Fore.GREEN + domain.get_typo3_version() + Fore.RESET)
+		print(' | known vulnerabilities:'.ljust(30) + Fore.GREEN + 'http://www.cvedetails.com/version-search.php?vendor=&product=Typo3&version=' + domain.get_typo3_version() + Fore.RESET)
+		print('')
+
+	def interesting_headers(name, value):
+		string = '[!] ' + name + ':'
+		print(string.ljust(30) + value)
+
+	def extension_output(extens):
+		if not extens:
+			print(Fore.RED + ' | No extension found' + Fore.RESET)
+		else:
+			for extension in extens:
+				print(Fore.BLUE + '\n[+] Name: ' + extension.split('/')[3] + '\n' + "-"* 25  + Fore.RESET)
+				print(' | Location:'.ljust(16) + extension)
+				if not (extens[extension] == False):
+					print(' | ' + extens[extension].split('.')[0] + ':'.ljust(4) + (extension + '/'+ extens[extension]))

lib/privoxy_only.py

@@ -1,56 +1,76 @@
+#-------------------------------------------------------------------------------
+# Typo3 Enumerator - Automatic Typo3 Enumeration Tool
+# Copyright (c) 2015 Jan Rude
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/](http://www.gnu.org/licenses/)
+#-------------------------------------------------------------------------------
+
 import socket
-import urllib2
 import os, sys
 import re
-
+from colorama import Fore
+from lib.request import Request
 try:
 	import socks
 except:
-	print "The module 'SocksiPy' is not installed."
+	print(Fore.RED + 'The module \'SocksiPy\' is not installed.')
 	if sys.platform.startswith('linux'):
-		"Please install it with: sudo apt-get install python-socksipy"
+		print('Please install it with: sudo apt-get install python-socksipy' + Fore.RESET)
 	else:
-		"You can download it from http://socksipy.sourceforge.net/"
+		print('You can download it from http://socksipy.sourceforge.net/' + Fore.RESET)
 	sys.exit(-2)
 
-def start_daemon():
+class Privoxy:
-	if sys.platform.startswith('linux'):
+	def __init__(self, port=8118):
-		os.system('service privoxy start')
+		self.__port = port
-		print '[ ok ] Starting privoxy daemon...done.'
-	elif sys.platform.startswith('win32') or sys.platform.startswith('cygwin'):
-		print "Please make sure Privoxy is running..."
-	else:
-		print "You are using", sys.platform, ", which is not supported (yet)."
-		sys.exit(-2)
-	
-# Using Privoxy for all connections
-def connect(port):
-	print "\nChecking connection..."
-	socks.setdefaultproxy(socks.PROXY_TYPE_HTTP, "127.0.0.1", port, True)
-	socket.socket = socks.socksocket
-	try:
-		url = urllib2.Request('https://check.torproject.org/')
-		torcheck = urllib2.urlopen(url)
-		response = torcheck.read()
-		torcheck.close()
-	except:
-		print "Failed to connect through Privoxy!"
-		print "Please make sure your configuration is right!\n"
-		sys.exit(-2)
-	try:
-		# TODO: Check on privoxy at http://ha.ckers.org/weird/privoxy.html
-		regex = re.compile("(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})")
-		searchIP = regex.search(response)
-		IP = searchIP.groups()[0]
-		print "Your IP is: ", IP
-	except:
-		print "It seems like Privoxy is not used.\nAborting...\n"
-		sys.exit(-2)
 
-def stop():
+	def start_daemon(self):
-	print "\n"
+		if sys.platform.startswith('linux'):
-	if sys.platform.startswith('linux'):
+			os.system('service privoxy start')
-		os.system('service privoxy stop')
+			print('[ ok ] Starting privoxy daemon...done.')
-		print '[ ok ] Stopping privoxy daemon...done.'
+		elif sys.platform.startswith('win32') or sys.platform.startswith('cygwin'):
-	elif sys.platform.startswith('win32') or sys.platform.startswith('cygwin'):
+			print('Please make sure Privoxy is running...')
-		print "You can close Privoxy now..."
+		else:
+			print('You are using', sys.platform, ', which is not supported (yet).')
+			sys.exit(-2)
+		
+	# Using Privoxy for all connections
+	def connect(self):
+		print('\nChecking connection...')
+		socks.setdefaultproxy(socks.PROXY_TYPE_HTTP, '127.0.0.1', self.__port, True)
+		socket.socket = socks.socksocket
+		try:
+			request = Request.get_request('https://check.torproject.org/')
+			response = request[1]
+		except:
+			print('Failed to connect through Privoxy!')
+			print('Please make sure your configuration is right!\n')
+			sys.exit(-2)
+		try:
+			# TODO: Check on privoxy at http://ha.ckers.org/weird/privoxy.html
+			regex = re.compile("(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})")
+			searchIP = regex.search(response)
+			IP = searchIP.groups()[0]
+			print('Your IP is: ', IP)
+		except:
+			print('It seems like Privoxy is not used.\nAborting...\n')
+			sys.exit(-2)
+
+	def stop(self):
+		print('\n')
+		if sys.platform.startswith('linux'):
+			os.system('service privoxy stop')
+			print('[ ok ] Stopping privoxy daemon...done.')
+		elif sys.platform.startswith('win32') or sys.platform.startswith('cygwin'):
+			print('You can close Privoxy now...')

lib/request.py

@@ -0,0 +1,83 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+#-------------------------------------------------------------------------------
+# Typo3 Enumerator - Automatic Typo3 Enumeration Tool
+# Copyright (c) 2015 Jan Rude
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/](http://www.gnu.org/licenses/)
+#-------------------------------------------------------------------------------
+
+import requests
+import re
+from colorama import Fore
+requests.packages.urllib3.disable_warnings()
+from lib.output import Output
+
+class Request:
+	"""
+	This class is used to make all server requests
+	"""
+	@staticmethod
+	def get_request(domain_name, path):
+		try:
+			r = requests.get(domain_name + path, timeout=10, headers={'User-Agent' : "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"}, verify=False)
+			httpResponse = r.text
+			headers = r.headers
+			cookies = r.cookies
+			status_code = r.status_code
+			response = [httpResponse, headers, cookies, status_code]
+			return response
+		except requests.exceptions.Timeout:
+			print(Fore.RED + '[x] Connection timed out' + Fore.RESET)
+		except requests.exceptions.ConnectionError as e: 
+			print(Fore.RED + '[x] Connection aborted.\n    Please make sure you provided the right URL' + Fore.RESET)
+		except requests.exceptions.RequestException as e:
+			print(Fore.RED + str(e) + Fore.RESET)
+
+	@staticmethod
+	def head_request(domain_name, path):
+		try:
+			r = requests.head(domain_name + path, timeout=10, headers={'User-Agent' : "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"}, allow_redirects=False, verify=False)
+			status_code = str(r.status_code)
+			if status_code == '405':
+				print("WARNING, (HEAD) method not allowed!!")
+				exit(-1)
+			return status_code
+		except requests.exceptions.Timeout:
+			print(Fore.RED + '[x] Connection timed out' + Fore.RESET)
+		except requests.exceptions.ConnectionError as e: 
+			print(Fore.RED + '[x] Connection aborted.\n    Please make sure you provided the right URL' + Fore.RESET)
+		except requests.exceptions.RequestException as e:
+			print(Fore.RED + str(e) + Fore.RESET)
+
+	@staticmethod
+	def interesting_headers(headers):
+		for header in headers:
+			if header == 'server':
+				Output.interesting_headers('Server', headers.get('server'))
+			elif header == 'x-powered-by':
+				Output.interesting_headers('X-Powered-By', headers.get('x-powered-by'))
+			elif header == 'via':
+				Output.interesting_headers('Via', headers.get('via'))
+
+	@staticmethod
+	# not used atm because unreliable
+	def version_information(domain_name, path, regex):
+		r = requests.get(domain_name + path, stream=True, timeout=10, headers={'User-Agent' : "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"}, verify=False)
+		if r.status_code == 200:
+			for content in r.iter_content(chunk_size=400, decode_unicode=False):
+				regex = re.compile(regex)
+				search = regex.search(str(content))
+				version = search.groups()[0]
+				return version

lib/settings.py

@@ -1,97 +0,0 @@
-#!/usr/bin/env python
-
-"""
-Copyright (c) 2014 Jan Rude
-"""
-
-from Queue import Queue
-from threading import Thread, Lock
-
-# Domain to check
-DOMAIN = ''
-
-# Maximum number of threads (avoiding connection issues and/or DoS)
-MAX_NUMBER_OF_THREADS = 10
-
-# Default port used by Tor
-DEFAULT_TOR_PORT = 9050
-
-# Default ports used in Tor proxy bundles
-DEFAULT_PRIVOXY_PORT = 8118
-
-# Default extension file
-# Default: all available Typo3 extensions
-EXTENSION_FILE = ['all_extensions']
-
-# List with selected extensions
-EXTENSION_LIST = []
-
-# List with extensions, where no versioninformation is available
-NO_VERSIONINFO = ['wt_spamshield', 'introduction'] #introduction has ChangeLog, but will use "Typo3 4.5.0" as version info!
-
-# Check only top X extensions
-# Default: all extensions
-TOP_EXTENSION = 'all'
-
-# HTTP User-Agent header value. Useful to fake the HTTP User-Agent header value at each HTTP request
-# Default: Mozilla/5.0
-user_agent = {'User-Agent' : "Mozilla/5.0"}
-
-# Maximum number of concurrent HTTP(s) requests (handled with Python threads)
-# Default: 7
-THREADS = 7
-
-# Verbosity.
-verbose = False
-
-#Input queue
-in_queue = ''
-
-# Seconds to wait before timeout connection.
-# Default: 20
-TIMEOUT = 20
-
-# Possible paths to Typo3 loginpage !! not used atm !!
-LOGIN_PATHS = ()
-
-# Possible paths and regex to typo3 version information
-TYPO3_VERSION_INFO = {'/typo3_src/ChangeLog':'RELEASE] Release of TYPO3 (.*)', '/typo3_src/NEWS.txt':'http://wiki.typo3.org/TYPO3_(\d{1,2}\.\d{1,2})', '/typo3_src/NEWS.md':"(.*) - WHAT'S NEW", 
-'/ChangeLog':'RELEASE] Release of TYPO3 (.*)', '/NEWS.txt':'http://wiki.typo3.org/TYPO3_(\d{1,2}\.\d{1,2})', '/NEWS.md':"(.*) - WHAT'S NEW"}
-
-# Typo3 verision details
-TYPO_VERSION = None
-
-# Possible paths to an extension
-EXTENSION_PATHS = ('/typo3conf/ext/', '/typo3/sysext/')
-
-# Possible version info file
-EXTENSION_VERSION_INFO = ('ChangeLog', 'README.txt')
-
-# Installed extensions on targed domain
-EXTENSIONS_FOUND = 0
-
-# Colorama is not used, if not installed
-COLORAMA = False
-
-
-
-## Not used atm ##
-
-# Maximum total number of redirections (regardless of URL) - before assuming we're in a loop
-MAX_TOTAL_REDIRECTIONS = 10
-
-# Maximum number of connection retries (to prevent problems with recursion)
-MAX_CONNECT_RETRIES = 100
-
-# Delay in seconds between each HTTP request.
-# Valid: float
-# Default: 0
-delay = 0
-
-# Maximum number of retries when the HTTP connection timeouts.
-# Valid: integer
-# Default: 3
-retries = 3
-
-# Use persistent HTTP(s) connections.
-KEEPALIVE = False

lib/start.py

@@ -1,89 +0,0 @@
-#!/usr/bin/env python
-
-"""
-Copyright (c) 2014 Jan Rude
-"""
-
-import time 
-from Queue import Queue
-from os.path import isfile
-from threading import Thread, Lock
-try:
-	from colorama import Fore, Back
-except:
-	pass
-from lib import settings
-from lib import versioninfo
-from lib import login
-from lib import output
-from lib import extensions
-
-# Startmethod
-def check_typo_installation(domain):
-	settings.DOMAIN = domain
-	settings.EXTENSIONS_FOUND = 0
-	if settings.COLORAMA:
-		output = Fore.CYAN + '[ Checking ' + domain + ' ]' + '\n' + "-"* 70  + Fore.RESET
-	else:
-		output = '[ Checking ' + domain + ' ]' + '\n' + "-"* 70
-	print '\n\n' + output
-
-	check = login.search_login()
-	if check is "redirect":
-		check_typo_installation(settings.DOMAIN)
-
-	elif check is True:
-		init_extension_search()
-	else:
-		mainpage = login.check_main_page()
-		if mainpage is True:
-			init_extension_search()
-		elif mainpage is not "skip":
-			output("Typo3 Login:".ljust(32) + "Typo3 is not used on this domain", False)
-
-def init_extension_search():
-	settings.in_queue = Queue()
-	versioninfo.search_version_info()
-	versioninfo.output()
-
-	if settings.TOP_EXTENSION != 0:
-		if not settings.EXTENSION_LIST:
-			extensions.generate_list()
-
-		extensions.copy()
-		extensions_to_check = settings.in_queue.qsize()
-
-		print '\nChecking', extensions_to_check, 'extension(s)...'
-		# Thanks to 'RedSparrow': http://stackoverflow.com/questions/17991033/python-cant-kill-main-thread-with-keyboardinterrupt
-		try:
-			while True:
-				if settings.in_queue.empty() == False:
-					time.sleep(0.5)
-					for i in xrange(0, settings.THREADS):
-						t = Thread(target=extensions.check_extension, args=())
-						t.daemon = True
-						t.start()
-				else:
-					break
-		except KeyboardInterrupt:
-			output("\nReceived keyboard interrupt.\nQuitting...", False)
-			exit(-1)
-		settings.in_queue.join()
-
-		installed_ext = settings.EXTENSIONS_FOUND
-		if installed_ext == 0:
-			output("No extensions installed", False)
-		else:
-			output('\n' + str(settings.EXTENSIONS_FOUND) + '/' + str(extensions_to_check) + ' extension(s) installed', True)
-	else:
-		print '\nSkipping check for extensions...'
-
-# print error messages
-def output(message, setting):
-	if settings.COLORAMA:
-		if not setting:
-			print Fore.RED + message + Fore.RESET
-		if setting:
-			print Fore.GREEN + message + Fore.RESET
-	else:
-		print message

lib/thread_pool.py

@@ -0,0 +1,132 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+#-------------------------------------------------------------------------------
+# Typo3 Enumerator - Automatic Typo3 Enumeration Tool
+# Copyright (c) 2015 Jan Rude
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/](http://www.gnu.org/licenses/)
+#-------------------------------------------------------------------------------
+
+import threading
+from queue import Queue
+
+class ThreadPoolSentinel:
+	pass
+
+class ThreadPool:
+	"""
+	Generic Thread Pool used for searching extensions and changelog/readme.
+	Any found extension or changelog/readme goes to the result queue:
+		work_queue:				Working queue
+		result_queue:			Result queue
+		active_threads:			Number of active threads
+		thread_list:			List of worker threads
+	"""
+	def __init__(self):
+		self.__work_queue = Queue()
+		self.__result_queue = Queue()
+		self.__active_threads = 0
+		self.__thread_list = []
+
+	def add_job(self, job):
+		# Load job in queue
+		self.__work_queue.put(job)
+
+	def get_result(self):
+		active_threads = self.__active_threads
+		while (active_threads) or (not self.__result_queue.empty()):
+			result = self.__result_queue.get()
+			if isinstance(result, ThreadPoolSentinel): # One thread was done
+				active_threads -= 1
+				self.__result_queue.task_done()
+				continue
+
+			else: # Getting an actual result
+				self.__result_queue.task_done()
+				yield result
+
+	def start(self, threads, version_search=False):
+		if self.__active_threads:
+			raise Exception('Threads already started.')
+
+		if not version_search:
+			 # Create thread pool
+			for _ in range(threads):
+				worker = threading.Thread(
+					target=_work_function,
+					args=(self.__work_queue, self.__result_queue))
+				worker.start()
+				self.__thread_list.append(worker)
+				self.__active_threads += 1
+		else:
+			for _ in range(threads):
+				worker = threading.Thread(
+					target=_work_function_version,
+					args=(self.__work_queue, self.__result_queue))
+				worker.start()
+				self.__thread_list.append(worker)
+				self.__active_threads += 1
+
+		# Put sentinels to let the threads know when there's no more jobs
+		[self.__work_queue.put(ThreadPoolSentinel()) for worker in self.__thread_list]
+
+	def join(self): # Clean exit
+		self.__work_queue.join()
+		[worker.join() for worker in self.__thread_list]
+		self.__active_threads = 0
+		self.__result_queue.join()
+
+def _work_function(job_q, result_q):
+	"""Work function expected to run within threads."""
+	while True:
+		job = job_q.get()
+
+		if isinstance(job, ThreadPoolSentinel): # All the work is done, get out
+			result_q.put(ThreadPoolSentinel())
+			job_q.task_done()
+			break
+
+		function = job[0]
+		args = job[1]
+		try:
+			result = function(*args)
+		except Exception as e:
+			print(e)
+		else:
+			if result == ('301' or '200' or '403'):
+				result_q.put((job))
+		finally:
+			job_q.task_done()
+
+def _work_function_version(job_q, result_q):
+	"""Work function expected to run within threads."""
+	while True:
+		job = job_q.get()
+
+		if isinstance(job, ThreadPoolSentinel): # All the work is done, get out
+			result_q.put(ThreadPoolSentinel())
+			job_q.task_done()
+			break
+
+		function = job[0]
+		args = job[1]
+		try:
+			result = function(*args)
+		except Exception as e:
+			print(e)
+		else:
+			if result == ('200'):
+				result_q.put((job))
+		finally:
+			job_q.task_done()

lib/tor_only.py

@@ -1,58 +1,81 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+#-------------------------------------------------------------------------------
+# Typo3 Enumerator - Automatic Typo3 Enumeration Tool
+# Copyright (c) 2015 Jan Rude
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/](http://www.gnu.org/licenses/)
+#-------------------------------------------------------------------------------
+
 import socket
-import urllib2
 import os, sys
 import re
+from colorama import Fore
+from lib.request import Request
 
 try:
 	import socks
 except:
-	print "The module 'SocksiPy' is not installed."
+	print(Fore.RED + 'The module \'SocksiPy\' is not installed.')
 	if sys.platform.startswith('linux'):
-		"Please install it with: sudo apt-get install python-socksipy"
+		print('Please install it with: sudo apt-get install python-socksipy' + Fore.RESET)
 	else:
-		"You can download it from http://socksipy.sourceforge.net/"
+		print('You can download it from http://socksipy.sourceforge.net/' + Fore.RESET)
 	sys.exit(-2)
 
-def start_daemon():
+class Tor:
-	if sys.platform.startswith('linux'):
+	def __init__(self, port=9050):
-		os.system('service tor start')
+		self.__port = port
-	elif sys.platform.startswith('win32') or sys.platform.startswith('cygwin'):
-		print "Please make sure TOR is running..."
-	else:
-		print "You are using", sys.platform, ", which is not supported (yet)."
-		sys.exit(-2)
-	
-# Using TOR for all connections
-def connect(port):
-	print "\nChecking connection..."
-	socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", port, True)
-	socket.socket = socks.socksocket
-	try:
-		url = urllib2.Request('https://check.torproject.org/')
-		torcheck = urllib2.urlopen(url)
-		response = torcheck.read()
-		torcheck.close()
-	except Exception, e:
-		print e
-		print "Failed to connect through TOR!"
-		print "Please make sure your configuration is right!\n"
-		sys.exit(-2)
-	try:
-		regex = re.compile('Congratulations. This browser is configured to use Tor.')
-		searchVersion = regex.search(response)
-		version = searchVersion.groups()
-		print "Connection to TOR established"
-		regex = re.compile("(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})")
-		searchIP = regex.search(response)
-		IP = searchIP.groups()[0]
-		print "Your IP is: ", IP
-	except:
-		print "It seems like TOR is not used.\nAborting...\n"
-		sys.exit(-2)
 
-def stop():
+	def start_daemon(self):
-	print "\n"
+		if sys.platform.startswith('linux'):
-	if sys.platform.startswith('linux'):
+			os.system('service tor start')
-		os.system('service tor stop')
+		elif sys.platform.startswith('win32') or sys.platform.startswith('cygwin'):
-	elif sys.platform.startswith('win32') or sys.platform.startswith('cygwin'):
+			print('Please make sure TOR is running...')
-		print "You can close TOR now..."
+		else:
+			print('You are using', sys.platform, ', which is not supported (yet).')
+			sys.exit(-2)
+		
+	# Using TOR for all connections
+	def connect(self):
+		print('\nChecking connection...')
+		socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, '127.0.0.1', self.__port, True)
+		socket.socket = socks.socksocket
+		try:
+			request = Request.get_request('https://check.torproject.org/')
+			response = request[1]
+		except Exception, e:
+			print(e)
+			print('Failed to connect through TOR!')
+			print('Please make sure your configuration is right!\n')
+			sys.exit(-2)
+		try:
+			regex = re.compile('Congratulations. This browser is configured to use Tor.')
+			searchVersion = regex.search(response)
+			version = searchVersion.groups()
+			pprint('Connection to TOR established')
+			regex = re.compile("(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})")
+			searchIP = regex.search(response)
+			IP = searchIP.groups()[0]
+			print('Your IP is: ', IP)
+		except:
+			print('It seems like TOR is not used.\nAborting...\n')
+			sys.exit(-2)
+
+	def stop(self):
+		print('\n')
+		if sys.platform.startswith('linux'):
+			os.system('service tor stop')
+		elif sys.platform.startswith('win32') or sys.platform.startswith('cygwin'):
+			print('You can close TOR now...')

lib/tor_with_privoxy.py

@@ -1,62 +1,84 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+#-------------------------------------------------------------------------------
+# Typo3 Enumerator - Automatic Typo3 Enumeration Tool
+# Copyright (c) 2015 Jan Rude
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/](http://www.gnu.org/licenses/)
+#-------------------------------------------------------------------------------
+
 import socket
-import urllib2
+import requests
 import os, sys
 import re
-
+from colorama import Fore
+from lib.request import Request
 try:
 	import socks
 except:
-	print "The module 'SocksiPy' is not installed."
+	print(Fore.RED + 'The module \'SocksiPy\' is not installed.')
 	if sys.platform.startswith('linux'):
-		"Please install it with: sudo apt-get install python-socksipy"
+		print('Please install it with: sudo apt-get install python-socksipy' + Fore.RESET)
 	else:
-		"You can download it from http://socksipy.sourceforge.net/"
+		print('You can download it from http://socksipy.sourceforge.net/' + Fore.RESET)
 	sys.exit(-2)
 
-def start_daemon():
+class Tor_with_Privoxy:
-	if sys.platform.startswith('linux'):
+	def __init__(self, port=8118):
-		os.system('service tor start')
+		self.__port = port
-		os.system('service privoxy start')
-		print '[ ok ] Starting privoxy daemon...done.'
-	elif sys.platform.startswith('win32') or sys.platform.startswith('cygwin'):
-		print "Please make sure TOR and Privoxy are running..."
-	else:
-		print "You are using", sys.platform, ", which is not supported (yet)."
-		sys.exit(-2)
-	
-# Using Privoxy and TOR for all connections
-def connect(port):
-	print "\nChecking connection..."
-	socks.setdefaultproxy(socks.PROXY_TYPE_HTTP, "127.0.0.1", port, True)
-	socket.socket = socks.socksocket
-	try:
-		url = urllib2.Request('https://check.torproject.org/')
-		torcheck = urllib2.urlopen(url)
-		response = torcheck.read()
-		torcheck.close()
-	except:
-		print "Failed to connect through Privoxy and/or TOR!"
-		print "Please make sure your configuration is right!\n"
-		sys.exit(-2)
-	try:
-		regex = re.compile('Congratulations. This browser is configured to use Tor.')
-		searchVersion = regex.search(response)
-		version = searchVersion.groups()
-		print "Connection to TOR established"
-		regex = re.compile("(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})")
-		searchIP = regex.search(response)
-		IP = searchIP.groups()[0]
-		print "Your IP is: ", IP
-	except Exception, e:
-		print e
-		print "It seems like TOR is not used.\nAborting...\n"
-		sys.exit(-2)
 
-def stop():
+	def start_daemon(self):
-	print "\n"
+		if sys.platform.startswith('linux'):
-	if sys.platform.startswith('linux'):
+			os.system('service tor start')
-		os.system('service tor stop')
+			os.system('service privoxy start')
-		os.system('service privoxy stop')
+			print('[ ok ] Starting privoxy daemon...done.')
-		print '[ ok ] Stopping privoxy daemon...done.'
+		elif sys.platform.startswith('win32') or sys.platform.startswith('cygwin'):
-	elif sys.platform.startswith('win32') or sys.platform.startswith('cygwin'):
+			print('Please make sure TOR and Privoxy are running...')
-		print "You can close TOR and Privoxy now..."
+		else:
+			print('You are using', sys.platform, ', which is not supported (yet).')
+			sys.exit(-2)
+		
+	# Using Privoxy and TOR for all connections
+	def connect(self):
+		print('\nChecking connection...')
+		socks.setdefaultproxy(socks.PROXY_TYPE_HTTP, "127.0.0.1", self.__port, True)
+		socket.socket = socks.socksocket
+		try:
+			request = Request.get_request('https://check.torproject.org/')
+			response = request[1]
+		except:
+			print('Failed to connect through Privoxy and/or TOR!')
+			print('Please make sure your configuration is right!\n')
+			sys.exit(-2)
+		try:
+			regex = re.compile('Congratulations. This browser is configured to use Tor.')
+			searchVersion = regex.search(response)
+			version = searchVersion.groups()
+			print('Connection to TOR established')
+			regex = re.compile("(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})")
+			searchIP = regex.search(response)
+			IP = searchIP.groups()[0]
+			print('Your IP is: ', IP)
+		except Exception as e:
+			print('It seems like TOR is not used.\nAborting...\n')
+			sys.exit(-2)
+
+	def stop(self):
+		print('\n')
+		if sys.platform.startswith('linux'):
+			os.system('service tor stop')
+			os.system('service privoxy stop')
+			print('[ ok ] Stopping privoxy daemon...done.')
+		elif sys.platform.startswith('win32') or sys.platform.startswith('cygwin'):
+			print('You can close TOR and Privoxy now...')

lib/update.py

@@ -1,105 +1,132 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
+#-------------------------------------------------------------------------------
+# Typo3 Enumerator - Automatic Typo3 Enumeration Tool
+# Copyright (c) 2015 Jan Rude
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/](http://www.gnu.org/licenses/)
+#-------------------------------------------------------------------------------
 
-import os, sys, gzip, urllib
+import os, sys, gzip, urllib.request
 from collections import OrderedDict
 import xml.etree.ElementTree as ElementTree
 
-# Progressbar
+class Update:
-def dlProgress(count, blockSize, totalSize):
+	"""
-	percent = int(count*blockSize*100/totalSize)
+	This class updates the Typo3 extensions
-	sys.stdout.write("\r[+] Downloading extentions: " + "%d%%" % percent)
-	sys.stdout.flush()
 
-# Download extensions from typo3 repository
+	It will download the extension file from the official repository, 
-def download_ext():
+	unpack it and sort the extensions in different files
-	try:
+	"""
-		urllib.URLopener.version = 'TYPO3/7.0.0'		
+	def __init__(self):
-		urllib.urlretrieve('http://ter.sitedesign.dk/ter/extensions.xml.gz', 'extensions.gz', reporthook=dlProgress)
+		self.download_ext()
-		inf = gzip.open('extensions.gz', 'rb')
+		self.generate_list()
-		file_content = inf.read()
-		inf.close()
-		outF = file("extensions.xml", 'wb')
-		outF.write(file_content)
-		outF.close()
-		os.remove('extensions.gz')
-	except Exception, e:
-		print "\nOops! Got:", e
 
-# Parse extensions.xml and save extensions in files
+	# Progressbar
-def generate_list():
+	def dlProgress(self, count, blockSize, totalSize):
-	experimental = {} # everything with 'experimental' and 'test'
+		percent = int(count*blockSize*100/totalSize)
-	alpha = {}
+		sys.stdout.write("\r[+] Downloading extentions: " + "%d%%" % percent)
-	beta = {}
+		sys.stdout.flush()
-	stable = {}
-	outdated = {} # everything with 'obsolete' and 'outdated'
-	allExt = {}
 
-	print "\n[+] Parsing file..."
+	# Download extensions from typo3 repository
-	tree = ElementTree.parse('extensions.xml') 
+	def download_ext(self):
-	root = tree.getroot()
+		try:	
-	extension = 0
+			urllib.request.urlretrieve('http://ter.sitedesign.dk/ter/extensions.xml.gz', 'extensions.gz', reporthook=self.dlProgress)
-	# for every extension in file
+			with gzip.open('extensions.gz', 'rb') as f:
-	for child in root:
+				file_content = f.read()
-		# insert every extension in "allExt" dictionary
+			f.close()
-		allExt.update({child.get('extensionkey'):child[0].text})
+			outF = open("extensions.xml", 'wb')
-		# and search the last version entry
+			outF.write(file_content)
-		version = 0
+			outF.close()
-		for version_entry in root[extension].iter('version'):
+			os.remove('extensions.gz')
-			version +=1
+		except Exception as e:
-		# get the state of the latest version
+			print ('\n', e)
-		state = (str(root[extension][version][2].text)).lower()
-		if state == 'experimental' or state == 'test':
-			experimental.update({child.get('extensionkey'):child[0].text})
-		elif state == 'alpha':
-			alpha.update({child.get('extensionkey'):child[0].text})
-		elif state == 'beta':
-			beta.update({child.get('extensionkey'):child[0].text})
-		elif state == 'stable':
-			stable.update({child.get('extensionkey'):child[0].text})
-		elif state == 'obsolete' or state == 'outdated':
-			outdated.update({child.get('extensionkey'):child[0].text})
-		extension+=1
 
-	# sorting lists according to number of downloads
+	# Parse extensions.xml and save extensions in files
-	print "[+] Sorting according to number of downloads..."
+	def generate_list(self):
-	sorted_experimental = sorted(experimental.iteritems(), key=lambda x: int(x[1]), reverse=True)
+		experimental = {} # 'experimental' and 'test'
-	sorted_alpha = sorted(alpha.iteritems(), key=lambda x: int(x[1]), reverse=True)
+		alpha = {}
-	sorted_beta = sorted(beta.iteritems(), key=lambda x: int(x[1]), reverse=True)
+		beta = {}
-	sorted_stable = sorted(stable.iteritems(), key=lambda x: int(x[1]), reverse=True)
+		stable = {}
-	sorted_outdated = sorted(outdated.iteritems(), key=lambda x: int(x[1]), reverse=True)
+		outdated = {} # 'obsolete' and 'outdated'
-	sorted_allExt = sorted(allExt.iteritems(), key=lambda x: int(x[1]), reverse=True)
+		allExt = {}
 
-	print "[+] Generating files..."
+		print ("\n[+] Parsing file...")
-	f = open(os.path.join('extensions', 'experimental_extensions'),'w')
+		tree = ElementTree.parse('extensions.xml') 
-	for i in xrange(0,len(sorted_experimental)):
+		root = tree.getroot()
-		f.write(sorted_experimental[i][0]+'\n')
+		extension = 0
-	f.close()
+		# for every extension in file
+		for child in root:
+			# insert every extension in "allExt" dictionary
+			allExt.update({child.get('extensionkey'):child[0].text})
+			# and search the last version entry
+			version = 0
+			for version_entry in root[extension].iter('version'):
+				version +=1
+			# get the state of the latest version
+			state = (str(root[extension][version][2].text)).lower()
+			if state == 'experimental' or state == 'test':
+				experimental.update({child.get('extensionkey'):child[0].text})
+			elif state == 'alpha':
+				alpha.update({child.get('extensionkey'):child[0].text})
+			elif state == 'beta':
+				beta.update({child.get('extensionkey'):child[0].text})
+			elif state == 'stable':
+				stable.update({child.get('extensionkey'):child[0].text})
+			elif state == 'obsolete' or state == 'outdated':
+				outdated.update({child.get('extensionkey'):child[0].text})
+			extension+=1
 
-	f = open(os.path.join('extensions', 'alpha_extensions'),'w')
+		# sorting lists according to number of downloads
-	for i in xrange(0,len(sorted_alpha)):
+		print ("[+] Sorting according to number of downloads...")
-		f.write(sorted_alpha[i][0]+'\n')
+		sorted_experimental = sorted(experimental.items(), key=lambda x: int(x[1]), reverse=True)
-	f.close()
+		sorted_alpha = sorted(alpha.items(), key=lambda x: int(x[1]), reverse=True)
+		sorted_beta = sorted(beta.items(), key=lambda x: int(x[1]), reverse=True)
+		sorted_stable = sorted(stable.items(), key=lambda x: int(x[1]), reverse=True)
+		sorted_outdated = sorted(outdated.items(), key=lambda x: int(x[1]), reverse=True)
+		sorted_allExt = sorted(allExt.items(), key=lambda x: int(x[1]), reverse=True)
 
-	f = open(os.path.join('extensions', 'beta_extensions'),'w')
+		print ("[+] Generating files...")
-	for i in xrange(0,len(sorted_beta)):
+		f = open(os.path.join('extensions', 'experimental_extensions'),'w')
-		f.write(sorted_beta[i][0]+'\n')
+		for i in range(0,len(sorted_experimental)):
-	f.close()
+			f.write(sorted_experimental[i][0]+'\n')
+		f.close()
 
-	f = open(os.path.join('extensions', 'stable_extensions'),'w')
+		f = open(os.path.join('extensions', 'alpha_extensions'),'w')
-	for i in xrange(0,len(sorted_stable)):
+		for i in range(0,len(sorted_alpha)):
-		f.write(sorted_stable[i][0]+'\n')
+			f.write(sorted_alpha[i][0]+'\n')
-	f.close()
+		f.close()
 
-	f = open(os.path.join('extensions', 'outdated_extensions'),'w')
+		f = open(os.path.join('extensions', 'beta_extensions'),'w')
-	for i in xrange(0,len(sorted_outdated)):
+		for i in range(0,len(sorted_beta)):
-		f.write(sorted_outdated[i][0]+'\n')
+			f.write(sorted_beta[i][0]+'\n')
-	f.close()
+		f.close()
 
-	f = open(os.path.join('extensions', 'all_extensions'),'w')
+		f = open(os.path.join('extensions', 'stable_extensions'),'w')
-	for i in xrange(0,len(sorted_allExt)):
+		for i in range(0,len(sorted_stable)):
-		f.write(sorted_allExt[i][0]+'\n')
+			f.write(sorted_stable[i][0]+'\n')
-	f.close()
+		f.close()
 
-	print '[+] Loaded', len(sorted_allExt), 'extensions\n'
+		f = open(os.path.join('extensions', 'outdated_extensions'),'w')
-	os.remove('extensions.xml')
+		for i in range(0,len(sorted_outdated)):
+			f.write(sorted_outdated[i][0]+'\n')
+		f.close()
+
+		f = open(os.path.join('extensions', 'all_extensions'),'w')
+		for i in range(0,len(sorted_allExt)):
+			f.write(sorted_allExt[i][0]+'\n')
+		f.close()
+
+		print ('[+] Loaded', len(sorted_allExt), 'extensions\n')
+		os.remove('extensions.xml')

lib/version_information.py

@@ -0,0 +1,50 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+#-------------------------------------------------------------------------------
+# Typo3 Enumerator - Automatic Typo3 Enumeration Tool
+# Copyright (c) 2015 Jan Rude
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/](http://www.gnu.org/licenses/)
+#-------------------------------------------------------------------------------
+
+import re
+from lib.request import Request
+
+class VersionInformation:
+	"""
+	This class will search for version information
+	"""
+	def search_typo3_version(self, domain):
+		ChangeLog = {'/typo3_src/ChangeLog':'(\d{1,2}\.\d{1,2}\.?[0-9]?[0-9]?)',  
+					'/ChangeLog':'(\d{1,2}\.\d{1,2}\.?[0-9]?[0-9]?)'
+					}
+		News = {'/typo3_src/NEWS.txt':'http://wiki.typo3.org/TYPO3_(\d{1,2}\.\d{1,2})', 
+				'/typo3_src/NEWS.md':"TYPO3 CMS (\d{1,2}\.\d{1,2}) - WHAT'S NEW", 
+				'/NEWS.txt':'http://wiki.typo3.org/TYPO3_(\d{1,2}\.\d{1,2})', 
+				'/NEWS.md':"TYPO3 CMS (\d{1,2}\.\d{1,2}) - WHAT'S NEW"
+				}
+
+		version = 'could not determine'
+		for path, regex in ChangeLog.items():
+			response = Request.version_information(domain.get_name(), path, regex)
+			if not(response is None):
+				version = response
+				break
+		if version == 'could not determine':
+			for path, regex in News.items():
+				response = Request.version_information(domain.get_name(), path, regex)
+				if not(response is None):
+					version = response
+					break
+		domain.set_typo3_version(version)

lib/versioninfo.py

@@ -1,47 +0,0 @@
-#!/usr/bin/env python
-
-"""
-Copyright (c) 2014 Jan Rude
-"""
-
-import re
-import urllib2
-from lib import settings
-try:
-	from colorama import Fore
-except:
-	pass
-
-# Searching for Typo3 version
-def search_version_info():
-	for path, regex in settings.TYPO3_VERSION_INFO.iteritems():
-		try:
-			request = urllib2.Request(settings.DOMAIN + path, None, settings.user_agent)
-			response = urllib2.urlopen(request, timeout = settings.TIMEOUT)
-			news = response.read(700)
-			response.close()
-			regex = re.compile(regex)
-			search = regex.search(news)
-			version = search.groups()
-			if settings.TYPO_VERSION is None or (len('Typo3' + version[0]) > len(settings.TYPO_VERSION)):
-				settings.TYPO_VERSION = 'Typo3 ' + version[0]
-				return
-		except:
-			pass
-
-# Output of Typo3 version
-def output():
-	if settings.TYPO_VERSION is None:
-		print "Typo3 Version:".ljust(32)
-		if settings.COLORAMA:
-			print Fore.RED 
-		print "Not found"
-		if settings.COLORAMA:
-			Fore.RESET
-	else:
-		if settings.COLORAMA:
-			output = Fore.GREEN + settings.TYPO_VERSION + Fore.RESET
-		else:
-			output = settings.TYPO_VERSION
-		print "Typo3 Version:".ljust(32) + output
-		print "Link to vulnerabilities:".ljust(32) + "http://www.cvedetails.com/version-search.php?vendor=&product=Typo3&version=" + settings.TYPO_VERSION.split()[1]

typo3_enumerator.py

@@ -0,0 +1,158 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+#-------------------------------------------------------------------------------
+# Typo3 Enumerator - Automatic Typo3 Enumeration Tool
+# Copyright (c) 2015 Jan Rude
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/](http://www.gnu.org/licenses/)
+#-------------------------------------------------------------------------------
+
+__version__ = "0.4"
+__program__ = "Typo-Enumerator"
+__description__ = 'Automatic Typo3 enumeration tool'
+__author__ = "https://github.com/whoot"
+
+import sys
+import os.path
+import datetime
+import argparse
+from colorama import Fore, init, deinit
+
+import lib.configuration as Config
+from lib.check_installation import Typo3_Installation
+from lib.version_information import VersionInformation
+from lib.extensions import Extensions
+from lib.domain import Domain
+from lib.update import Update
+from lib.output import Output
+init()
+
+class Typo3:
+	def __init__(self):
+		self.__domain_list = []
+		self.__extensions = None
+
+	def run(self):
+		parser = argparse.ArgumentParser(usage='typoenum.py [options]', add_help=False)
+		group = parser.add_mutually_exclusive_group()
+		anonGroup = parser.add_mutually_exclusive_group()
+		group.add_argument('-f', '--file', dest='file')
+		group.add_argument('-d', '--domain', dest='domain', type=str, nargs='+')
+		group.add_argument('-u', '--update', dest='update', action='store_true')
+		parser.add_argument('--top', type=int, dest='top', metavar='VALUE')
+		parser.add_argument('--state', dest='ext_state', choices = ['all', 'experimental', 'alpha', 'beta', 'stable', 'outdated'], nargs='+', default = ['all'])
+		anonGroup.add_argument('--tor', help='using only TOR for connections', action='store_true')
+		anonGroup.add_argument('--privoxy', help='using only Privoxy for connections', action='store_true')
+		anonGroup.add_argument('--tp', help='using TOR and Privoxy for connections', action='store_true')
+		parser.add_argument('-p', '--port', dest='port', help='Port for TOR/Privoxy (default: 9050/8118)', type=int)
+
+		parser.add_argument( "-h", "--help", action="help")
+		args = parser.parse_args()
+
+		try:
+			if args.update:
+				Update()
+				return True
+
+			if args.tor:
+				from lib.tor_only import Tor
+				if args.port:
+					tor = Tor(args.port)
+				else:
+					tor = Tor()
+				tor.start_daemon()
+				tor.connect()
+
+			elif args.privoxy:
+				from lib.privoxy_only import Privoxy
+				if args.port:
+					privoxy = Privoxy(args.port)
+				else:
+					privoxy = Privoxy()
+				privoxy.start_daemon()
+				privoxy.connect()
+
+			elif args.tp:
+				from lib.tor_with_privoxy import Tor_with_Privoxy
+				if args.port:
+					tp = Tor_with_Privoxy(args.port)
+				else:
+					tp = Tor_with_Privoxy()
+				tp.start_daemon()
+				tp.connect()
+
+			if args.domain:
+				for dom in args.domain:
+					self.__domain_list.append(Domain(dom, args.ext_state, args.top))
+			elif args.file:
+				if not os.path.isfile(args.file):
+					print(Fore.RED + "\n[x] File not found: " + args.file + "\n |  Aborting..." + Fore.RESET)
+					sys.exit(-2)
+				else:
+					with open(args.file, 'r') as f:
+						for line in f:
+							domain_list.append(Domain(line.strip('\n'), args.ext_state, args.top))
+			for domain in self.__domain_list:
+				print('\n\n' + Fore.CYAN + '[ Checking ' + domain.get_name() + ' ]' + '\n' + "-"* 73  + Fore.RESET)
+				Typo3_Installation.run(domain)
+				if not domain.get_typo3():
+					print(Fore.RED + '\n[x] Typo3 is not used on this domain' + Fore.RESET)
+				else:
+					if domain.get_login_found():
+						# search version info
+						version = VersionInformation()
+						version.search_typo3_version(domain)
+					Output.typo3_installation(domain)
+					if not domain.get_login_found():
+						print(Fore.YELLOW + '[!] Backend login not found\n | Extension enumeration would be failing\n | Skipping...' + Fore.RESET)
+					else:
+						# Loading extensions
+						if (self.__extensions is None):
+							ext = Extensions(args.ext_state, args.top)
+							self.__extensions = ext.load_extensions()
+						# copy them in domain object
+						if (domain.get_extensions() is None):	
+							domain.set_extensions(self.__extensions)
+						# search
+						print ('\n[ Searching', len(self.__extensions), 'extensions ]')
+						ext.search_extension(domain, self.__extensions)
+						ext.search_ext_version(domain, domain.get_installed_extensions())
+						Output.extension_output(domain.get_installed_extensions())
+		
+		except KeyboardInterrupt:
+			print("\nReceived keyboard interrupt.\nQuitting...")
+			exit(-1)
+		finally:
+			deinit()
+			now = datetime.datetime.now()
+			print('\n\n' + __program__ + ' finished at ' + now.strftime("%Y-%m-%d %H:%M:%S") + '\n')
+		
+if __name__ == "__main__":
+	print('\n' + 73*'=')
+	print(Fore.BLUE + ' _______                     ______ '.center(73))
+	print('|_     _|.--.--.-----.-----.|__    |'.center(73))
+	print('  |   |  |  |  |  _  |  _  ||__    |'.center(73))
+	print('  |___|  |___  |   __|_____||______|'.center(73))
+	print('         |_____|__|                 '.center(73))
+	print(' _______                                         __              '.center(73))
+	print('|    ___|.-----.--.--.--------.-----.----.---.-.|  |_.-----.----.'.center(73)) 
+	print('|    ___||     |  |  |        |  -__|   _|  _  ||   _|  _  |   _|'.center(73))
+	print('|_______||__|__|_____|__|__|__|_____|__| |___._||____|_____|__|  '.center(73))
+	print(Fore.RESET)
+	print(__description__.center(73))
+	print(('Version ' + __version__).center(73))
+	print((__author__).center(73))
+	print(73*'=')
+	main = Typo3()
+	main.run()

typoenum.py

@@ -1,156 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-############ Version information ##############
-__version__ = "0.3.3"
-__program__ = "Typo-Enumerator v" + __version__
-__description__ = 'Automatic Typo3 and Typo3 extensions enumeration tool'
-__author__ = "Jan Rude"
-__licence__ = "BSD Licence"
-__status__ = "Development"
-###############################################
-
-import os
-import datetime
-import argparse
-import warnings
-warnings.filterwarnings(action="ignore", message=".*was already imported", category=UserWarning)
-from lib import settings
-from lib import update
-from lib import start
-try:
-	from colorama import init, Fore
-	settings.COLORAMA = True
-except:
-	pass
-init()
-
-
-# Main
-def main(argv):
-	parser = argparse.ArgumentParser(usage='typoenum.py [options]')
-	group = parser.add_mutually_exclusive_group()
-	anonGroup = parser.add_mutually_exclusive_group()
-	extensionGroup = parser.add_mutually_exclusive_group()
-	group.add_argument('-d', '--domain', dest='domain', type=str, nargs='+')
-	group.add_argument('-f', '--file', dest='file', help='File with a list of domains')
-	group.add_argument('-u', '--update', dest='update', action='store_true',help='Update the extension file')
-	parser.add_argument('--user_agent', dest='user_agent', metavar='USER-AGENT (default: Mozilla/5.0)')
-	extensionGroup.add_argument('--top', type=int, dest='top', metavar='VALUE', help='Check only most X downloaded extensions (default: all)')
-	extensionGroup.add_argument('-e', '--extension', type=str, dest='ext', metavar='EXTENSION', help='Check only for this extension(s)', nargs='+')
-	parser.add_argument('--state', dest='ext_state', help='Check only (experimental | alpha | beta | stable | outdated) extensions', nargs='+')
-	anonGroup.add_argument('--tor', help='using only TOR for connections', action='store_true')
-	anonGroup.add_argument('--privoxy', help='using only Privoxy for connections', action='store_true')
-	anonGroup.add_argument('--tp', help='using TOR and Privoxy for connections', action='store_true')
-	parser.add_argument('-p', '--port', dest='port', help='Port for TOR/Privoxy (default: 9050/8118)', type=int)
-	parser.add_argument('-t', '--threads', dest='threads', default=settings.THREADS, type=int, help=' Threads for HTTP connections (default: 7)')
-	parser.add_argument('--timeout', dest='timeout', default=settings.TIMEOUT, type=int, help='(default: 20)')
-	parser.add_argument('-v', '--verbose', help='increase output verbosity', action='store_true')
-	args = parser.parse_args()
-
-	try:
-		if args.update:
-			update.download_ext()
-			update.generate_list()
-			return True
-
-		if args.threads > settings.MAX_NUMBER_OF_THREADS:
-			output("Warning! Threads are set to", args.threads,"(max value is 10)\nThis can cause connection issues and/or DoS\nAborting....")
-			sys.exit(-2)
-
-		if args.tor:
-			from lib import tor_only
-			tor_only.start_daemon()
-			if args.port:
-				tor_only.connect(args.port)
-			else:
-				tor_only.connect(settings.DEFAULT_TOR_PORT)
-
-		elif args.privoxy:
-			from lib import privoxy_only
-			privoxy_only.start_daemon()
-			if args.port:
-				privoxy_only.connect(args.port)
-			else:
-				privoxy_only.connect(settings.DEFAULT_PRIVOXY_PORT)
-
-		elif args.tp:
-			from lib import tor_with_privoxy as tp
-			tp.start_daemon()
-			if args.port:
-				tp.connect(args.port)
-			else:
-				tp.connect(settings.DEFAULT_PRIVOXY_PORT)
-
-		if args.timeout:
-			settings.TIMEOUT = args.timeout
-
-		if args.user_agent:
-			settings.user_agent.update({'User-Agent':args.user_agent})
-
-		if args.verbose:
-			settings.verbose = args.verbose
-
-		if args.top or args.top is 0:
-			settings.TOP_EXTENSION = args.top
-
-		if args.ext:
-			for extension in args.ext:
-				settings.EXTENSION_LIST.append(extension)
-
-		if args.ext_state:
-			settings.EXTENSION_FILE = []
-			for ext_list in args.ext_state:
-				ext_file = ext_list + '_extensions'
-				settings.EXTENSION_FILE.append(ext_file)
-
-		if args.domain:
-			for dom in args.domain:
-				start.check_typo_installation(dom)
-
-		elif args.file:
-			if not isfile(args.file):
-				output("\nFile not found: " + args.file + "\nAborting...")
-				sys.exit(-2)
-			else:
-				with open(args.file, 'r') as f:
-					for line in f:
-						start.check_typo_installation(line.strip('\n')[0])
-	except KeyboardInterrupt:
-		output("\nReceived keyboard interrupt.\nQuitting...")
-		exit(-1)
-	except Exception, e:
-		import traceback
-		print ('generic exception: ', traceback.format_exc())
-
-	finally:
-		if args.tor:
-			tor_only.stop()
-
-		elif args.privoxy:
-			privoxy_only.stop()
-
-		elif args.tp:
-			tp.stop()
-
-		now = datetime.datetime.now()
-		print '\n' + __program__ + ' finished at ' + now.strftime("%Y-%m-%d %H:%M:%S") + '\n'
-		return True
-
-# print error messages
-def output(message):
-	if settings.COLORAMA:
-		print Fore.RED + message + Fore.RESET
-	else:
-		print message
-
-if __name__ == "__main__":
-	import sys
-	print('\n' + 70*'*')
-	print('\t' + __program__ )
-	print('\t' + __description__)
-	print('\t' + '(c)2014 by ' + __author__)
-	print('\t' + 'Status:\t' + __status__)
-	print('\t' + 'For legal purposes only!')
-	print(70*'*' + '\n')
-	sys.exit( not main( sys.argv ) )