1 to 25 of 126 Data Lake Jobs in London

Product Manager (Product Owner Business Analysis Data Lake Data Mesh Datamesh SQL Architecture Big Data AWS Python Dremio Apache Iceburg Iceberg Arrow DBT Tableau PowerBI Finance Trading Front Office Investment Banking Asset Manager Financial Services FX Fixed Income Equities Commodities Derivatives Hedge Fund … London. You MUST have the following: Advanced ability as a Product Owner/Product Manager/Business Aanlyst Extensive exposure to data lake environments Some exposure to data mesh architecture projects Some SQL querying Agile The following is DESIRABLE, not essential: Trading, Front Office finance … of data into the company from external financial data providers. The data is fed into an AWS based datalake and then distributed across the Front Office. This data lake will be migrated to a data mesh architecture using Dremio more »

right thingCreate opportunity - For our people to excelThe SZC Technical Lead duties are primarily focused on the performance of the SZC data lake as the core data management tool on the project, leading a small team of data engineers to ensure high quality … for:The SZC Technical Lead will understand and bring together the differing information streams available through its access to the SZC Data-lake, Enterprise software tools and other systems used to collect, migrate and manage data.Advising and supporting implementation and user acceptance testing on new software, as … lead include working closely with the IT function and with system administrators to aid data availability through the SZC Data-lake, using API techniques through Azure Data Factory and ETL techniques through Databricks.Defining and setting development, test, release, update, and support processes for more »

D365, D365 F&O, D365 Consultant, AX 2012, Dynamics AX, Microsoft power BI, SSRS, ETL, Reporting developer, SQL, Azure synapse, Data Lake, Lake House, Delta Lake, Data verse, DAX, BI reports, Paginated Report Builder, South London, Hybrid, £60-£75K Our end user client … with Microsoft Dynamics 365 Finance Operations and Project Operations 4+ years' experience working with Microsoft Power BI 4+ years' experience working with enterprise data warehouse/BI projects. … 4+ years' experience working with Data Analysis Expressions (DAX) Extensive experience with SQL Experience with Data Verse Data Lake, Lake House, and Delta Lake Experience Responsibilities: Dynamics Specific Development, Maintenance, and Support: Developing, maintaining, and providing support for custom analytics and more »

Data Technical Lead Location: Hybrid - London Package: Negotiable + Benefits Brown & Brown, Inc. is one of the largest insurance brokers globally having in excess of $4bn revenue in 2023 by providing risk management solutions to help protect what our customers value most. We have experienced significant growth over … size of the business by improving customer experience, growing organically and acquiring, accelerating knowledge and capabilities of teammates and using technology with purpose. Data and Data services is a key strategic enabler in Brown and Brown Europe as we look to integrate acquired business, enable them … Technical Skills – Deep hands on experience with the following data technologies: Stack including, Azure Data factory, Azure Data Lake Storage, Azure SQL, Azure Cosmos DB, Azure Power Platform, primarily power BI Data Lake & Data Mesh implementation The rewards more »

for designing, developing, and maintaining data pipelines and systems in Azure Cloud environments, with a strong emphasis on Azure Data Lake, Azure Data Factory, and Microservices architecture. Key Responsibilities: Develop and maintain data pipelines in Azure Cloud environments, leveraging Azure Data Lake, Azure Data Factory, and Microservices architecture. Design and implement scalable and efficient data processing solutions using technologies such as ServiceBus, REST API, App Service, SQL Data Warehouse, and Data Mart. Utilize Vector Databases, Cosmos DB, Redis, and Elasticsearch … engineering. Bachelor’s Degree or higher in Computer Science or a related field. Proficiency in Azure Cloud environments, Azure Data Lake, Azure Data Factory, Microservices architecture. Experience with Vector Databases, Cosmos DB, Redis, Elasticsearch. Strong programming skills in Python, Spark, Databricks, Pyspark, SQL more »

closely with Data Architecture, Data Management, The EDP product owner and the Technology team delivering the EDP. Data Lake Structure and Design: Validate the medallion model data lake structure (bronze, silver, gold) as the optimal approach and recommend alternatives if … needed. Create a Data Lake design including namespace hierarchy, and naming standards. Design to optimise storage and retrieval based on data types, usage patterns, and the emerging Data Domain segregation. Establish policies for moving data between different storage tiers within the … data lake (e.g. hot, warm, cold storage) based on access frequency and cost considerations. Establish policies for data integrity, consistency, conformance, and accuracy when transforming data from Bronze to Silver, and Silver to Bronze. Develop a Data Modelling Design, including a more »

Are you ready to take your data engineering expertise to new heights? This is your chance to become an integral part of a pioneering construction leader that's shaping the future! With a rich history, my client is an acclaimed construction powerhouse, renowned for iconic projects that transcend … data solutions aligned with business requirements. Design and fine-tune data storage and retrieval systems using Azure Data Lake Storage, Azure SQL Database, and other Azure DB services. Develop, optimize, and maintain data pipelines for automated data integration from … and/or BI Developer. Proficiency in Azure data tools such as Data Factory, Synapse, Databricks, SQL, Data Lake or Delta Lake. Strong data engineering and data modeling skills, with the ability to craft complex data models. more »

mission by providing a hugely diverse set of technical solutions and platforms. From supporting critical systems which underpin the UK economy, to evolving data analytics, reinventing our online presence, and introducing a fully digital workplace. Be part of a team that’s constantly evolving, just like our industry.We … deliver services to both the Bank and external users. These include critical systems and services that underpin the UK economy, financial supervision, evolving data analytics, collaboration and research.This role sits within the Technology Strategy and Architecture (TSA) division in the Technology Directorate who are accountable for technical strategy … and experience in architecture modelling toolsDeep domain expertise in cloud scale analytics and experience in one or more technologies e.g. Azure Data Lake, Azure Data Factory, Azure Databricks, Azure Machine Learning, Azure Synapse, Azure ML, Azure Cognitive Services, PowerBIKnowledge and exposure of financial regulation is more »

This is an exciting opportunity for a Data Architect to join a growing Data & AI team. The successful applicant will work alongside architects and consultants to deliver high-quality data solutions to help customers become more successful in becoming a data-guided … platforms and data solutions using a wide range of Azure Services including Azure Databricks, Azure SQL Database, Azure Data Lake Storage, Azure Synapse, Azure Cosmos DB, Azure Data Factory and Azure Blob storage Building conceptual, logical and physical data models … in Data Warehouse methodologies Experience working across Azure Data Services including Azure Databricks, Azure SQL Database, Azure Data Lake Storage, Azure Cosmos DB, Azure Data Factory, Azure Blob storage and Power BI Strong experience in SQL Good understanding of Azure infrastructure more »

will be invaluable. Main Responsibilities: Design and implement effective database solutions and models to store and retrieve data including Data Lake and Data Warehousing Assess database implementation procedures to ensure they comply with GDPR and data compliance Design conceptual and logical … the benefits, pros and cons of solution options. Skills & experience required: Strong hands-on experience in Data Warehouse and Data Lake technologies preferably around Azure Extensive experience in implementing solutions around DataBricks, Azure Data Factory, Azure SQL, CosmosDB, Cognite Services, Storage Accounts, Azure … of working with structured and unstructured datasets Able to design a range of architectural solutions Data models and ETL Data lake and data warehouse end to end architecture Experience within media, publishing, research, or a similar consumer focused industry is highly desirable, but more »

Data Analyst Location: Hybrid - London Package: Negotiable + Benefits Data and Data services is a key strategic enabler in Brown and Brown Europe as we look to integrate acquired business, enable them to provide great customer service and grow through the provision of superior … data services The Data Analyst for Data Operations will be responsible for managing stakeholders, gathering, collating and analysing data issues and translating them into data and reporting needs, using technical expertise to ensure data accuracy and high-quality … communication and collaboration skills Experience with data warehousing Experience in the Microsoft stack - Azure Data factory, Azure Data Lake Storage, Azure SQL Server, Azure Power Platform, Power BI. Working early some mornings, on a scheduled basis to monitor overnight operational batches and resolve more »

Requirements At least two years of professional experience as a data engineer with a proven ability to provide effective data solutions in a production setting. Knowledge of developing real-time data stream systems (ideally Kafka). Proven track record in developing data systems using PySpark and Apache Spark for batch processing. Capable of managing data intake from various sources, including data streams, unstructured data, relational databases, and NoSQL databases. Extensive knowledge in distributed systems, cloud architecture, and data pipelines. Proficiency in Python … making and client visibility. Play a major role in the development and optimisation of data systems to support our Data Lake/Data Mesh architecture and data processing needs, ensuring the provision of scalable, high-performance data products for more »

the opportunity to share in the success of our business. Job description: Department: RUTM - Corporate Location: London Contract Type: Permanent Reporting to: Business Data Manager Purpose Asset management domain specialist data analyst responsible for building out data workflows to support automation of foundational data domains, RAM business processes and reporting. There will be particular focus on leveraging tools such as Alteryx (low code data science platform), and PowerBI as part of the RAM data strategy to turn RAM into a data driven business. The Role Help … practices advantageous Ability to write SQL code and join data tables. Ability to leverage data from internal data lake and build data models in data tools Experience working with APIs The ability to work autonomously and as part of more »

be a developer in the Client Reporting Product Team within Distribution Technology. The Client Reporting Product team partners with Marketing, Reporting Operations and Data Management teams in Technology to build data integration solutions for the automation of a wide variety and high volume of reports. This … software development skills and can pick up new technologies and assist in all aspects of the team’s work such as development using data warehouse and master data vendor platform tools. You will not only need excellent problem-solving skills, but you should also be able … Informatica or similar ETL/ELT platforms, as well as knowledge of data warehousing, data mart, and data lake concepts and processes Work within the team as directed to automate and improve existing data designs and data models, including more »

decision making, deep technical know-how, and an ability to navigate complex business challenges while building and leading a team of world class data and analytics leaders.Are you passionate about Data and Analytics and excited about how it can completely transform the way an enterprise works … Do you have the strategic vision, technical expertise, and leadership skills to drive data-driven solutions? Do you want to work in a dynamic, fast-growing category? If so, you might be the ideal candidate for the role of Solution Architect Data Foundations, in the Enterprise … Architecture function for Global Pet Nutrition (PN) at Mars.The Solution Architect Data Foundations is a strategic leadership role that oversees delivery of cross product transversal data capabilities that is foundational to our success. This role is accountable for the architecture and design andoptimization of data more »

Lead Azure Data Engineer | PySpark (Python) & Synapse | Tech for Good/Charity Rate: £500-650 per day Duration: 6-12 months IR35: Outside Location: Remote with occasional travel to London (Once every two months max) Essential skills required: Azure – solid experience required of the Azure Data … ESSENTIAL as PySpark is used heavily. You will be tested on PySpark Azure Synapse - ESSENTIAL as is used heavily Spark Azure Data Lake/Data Bricks/Data Factory Be happy to act as a lead and mentor to the other permanent Azure … This is an urgent contract, so if you are interested apply ASAP. Lead Azure Data Engineer | PySpark (Python), Synapse, Data Lake | Tech for Good/Charity more »

Data Engineer Grade: Senior Consultant – Senior Manager Location: London Employment Type: Full Time The Opportunity: A Tier One Technology Consultancy seeks a highly ambitious and technically astute AWS Data Engineer to help design, build and deploy cloud-based data solutions - transforming raw data … in data cleaning, wrangling, visualization, and reporting. • Specialised in AWS cloud technologies for ETL, data warehouse, and data lake design. • Hands-on experience with AWS services like EMR, Glue, RedShift, Kinesis, Lambda, DynamoDB. • Capable of processing large volumes of structured and unstructured data … AWS cloud, including Platform-as-a-Service, server-less, and container technologies. • Diverse skills in ETL, data warehouse, and Data Lake design. • Familiarity with various tools, cloud technologies, and approaches. • Expertise in AWS or equivalent open-source tools. • Proficient in defining the end-to-end more »

Role: Data Modelling Job Type: Permanent Location: London (Hybrid) Ready to utilize your Data analysis and Investigatory skills ? Are you passionate about RDBMS systems and converting data requirements into logical models and physical solutions? We have an exciting role for you as a Data … the future of technology. with some of the brightest global minds in the industry. access to endless learning opportunities. The Role As a Data Modeler, you will utilise your analytical skills to design, implement and create conceptual, logical, and physical data models. Key responsibilities: modelling standards … to convert data requirements into logical models and physical solutions. with data warehousing solutions (e.g. Snowflake) and data lake architecture, Azure Databricks/Pyspark & ADF. retail data model standards - ADRM. communication skills, organisational and time management skills. to innovate, support change more »

do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … consecutive years Have practical experience designing and delivering using AWS or Azure such as:Azure SQL Data Warehouse, Azure Data Lake,AWS S3,AWS RDS,AWS Lambda or similar Have experience with Open Source big data products i.e.Hadoop Hive, Pig, Impala or similar … scripting languages such as Python, Java or similar Have expertise in data modelling, data warehouse design and data lake concepts and practices. Additional Information This role can be delivered in a hybrid nature from one of these offices: Belfast, Birmingham, Edinburgh, London, Manchester more »

AWS Data Architect (Must have current SC) Amber Labs was born out of the recognition that many organisations require support on their data maturity journey. Our team is passionate and experienced in delivering cutting-edge data capability. Amber Labs was established with a vision … to provide specialist services, automation, accelerators competitively to organisations across EMEA. Job Summary We are seeking a highly skilled AWS Data Architect to join our dynamic team. The ideal candidate will be responsible for designing, implementing, and optimizing our data architecture on the AWS platform. You … will work closely with various teams to ensure data integrity, security, and availability, driving our data strategy forward Key Responsibilities Design and Implement Data Solutions: Develop scalable and efficient data architecture solutions on AWS. Create and maintain data pipelines using more »

unparalleled educational experience. By attracting and retaining the finest educators worldwide, our client ensures an environment where excellence thrives. Job Purpose As the Data Platform Engineer overseeing the Data Warehouse and analytics infrastructure, you will shape the backbone of our client's business analysis and reporting … capabilities, both now and into the future. They are seeking a visionary with a proven track record of architecting enterprise-level data ecosystems, someone who thrives on pushing the boundaries of what's possible in data manipulation. Your expertise will ensure data is not … Remarkable experience with Development and maintenance of data pipeline and ETL processes using Azure data Factory, Azure Data Lake and Azure Databricks. Exceptional experience developing codes, testing for quality assurance, administering Azure Databricks platform and exposure to Unity Catalog. Exposure of Performance Tuning more »

Are you passionate about leveraging cutting-edge data technologies to drive business success? Do you thrive in dynamic environments where innovation and collaboration are valued? If so, we have an exciting opportunity for you! Position: Lead Data Engineer Responsibilities: Lead Data Management and Architecture … Spearhead the design and implementation of robust data solutions across the organization, ensuring scalability and efficiency. Next Gen Data Platform Leadership: Act as the primary data engineer supporting the development of our next-generation data platform, collaborating closely with third-party vendors … Skills and Experience: Proficiency in Microsoft Azure Data Services, including Azure SQL Database, Azure Data Factory, Azure Data Lake Storage, and Azure HDInsight. Advanced skills in SQL programming, with the ability to design, optimize, and maintain complex queries and databases. Strong experience in more »

partner with Deutsche Bank to support contingent recruitment processes. Job description - the role On behalf of Deutsche Bank, we are looking for a Data Governance Analyst for an initial 6 month contract based in the Banks London Office on a hybrid working model of 3 days per week … Together we're sharing new perspectives and transforming what it means to be a bank.' Purpose of the role: Transaction Monitoring & Screening Global Data Steward Office is responsible for Data Governance, Data Quality Management (DQM) & Data Programme delivery covering key functions within … Transaction Monitoring and Screening. Working as part of the DQM team the Senior Data Analyst will be responsible for supporting the Data Quality Management Framework and for contributing towards the central Data Analytics capability. Data Quality Management Framework covers Data more »

Data Migration Developer (SC Cleared) London (Hybrid) Permanent £7 0,000 - £90,000 (DOE) + Benefits A highly experienced Data Migration Developer wanted with active SC clearance.This fantastic opportunity will involve playing a dual role as a Migration Specialist and Developer. Hybrid working with the expectation … to write high-quality, maintainable code using languages such as Python, SQL Server. PC, PowerShell, Sharepoint, Azure, Desktop Support , Snowflake, Azure Data Lake services (ALA) Gen 2. Expertise in development, testing Proof of Concepts (PoCs), delivering Minimum Viable Products (MVPs) is essential for achieving project success. Develop … Plan, design, and execute PoCs to validate new technologies, frameworks, or solutions. Establish data tools like Snowflake and Azure Data Lake Services (ADLS) Gen 2. Utilize PowerBI, Tableau, or similar tools to design and create interactive and visually appealing dashboards and reports. Familiarity with data more »

Data Migration Developer (SC Cleared) London (Hybrid) Permanent £7 0,000 - £90,000 (DOE) + Benefits A highly experienced Data Migration Developer wanted with active SC clearance.This fantastic opportunity will involve playing a dual role as a Migration Specialist and Developer. Hybrid working with the expectation … to write high-quality, maintainable code using languages such as Python, SQL Server. PC, PowerShell, Sharepoint, Azure, Desktop Support , Snowflake, Azure Data Lake services (ALA) Gen 2. Expertise in development, testing Proof of Concepts (PoCs), delivering Minimum Viable Products (MVPs) is essential for achieving project success. Develop … Plan, design, and execute PoCs to validate new technologies, frameworks, or solutions. Establish data tools like Snowflake and Azure Data Lake Services (ADLS) Gen 2. Utilize PowerBI, Tableau, or similar tools to design and create interactive and visually appealing dashboards and reports. Familiarity with data more »